2023-03-08 18:53:34,190 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-08 18:53:34,486 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac 2023-03-08 18:53:34,506 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-08 18:53:34,507 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/cluster_8eed2209-15fd-42d6-1e88-8ecffcfc5d58, deleteOnExit=true 2023-03-08 18:53:34,507 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-08 18:53:34,508 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/test.cache.data in system properties and HBase conf 2023-03-08 18:53:34,509 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/hadoop.tmp.dir in system properties and HBase conf 2023-03-08 18:53:34,509 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/hadoop.log.dir in system properties and HBase conf 2023-03-08 18:53:34,510 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-08 18:53:34,510 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-08 18:53:34,510 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-08 18:53:34,632 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-08 18:53:35,065 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-08 18:53:35,070 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-08 18:53:35,071 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-08 18:53:35,071 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-08 18:53:35,071 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-08 18:53:35,072 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-08 18:53:35,072 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-08 18:53:35,072 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-08 18:53:35,073 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-08 18:53:35,073 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-08 18:53:35,074 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/nfs.dump.dir in system properties and HBase conf 2023-03-08 18:53:35,074 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/java.io.tmpdir in system properties and HBase conf 2023-03-08 18:53:35,074 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-08 18:53:35,075 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-08 18:53:35,075 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-08 18:53:35,681 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-08 18:53:35,685 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-08 18:53:35,983 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-08 18:53:36,165 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-08 18:53:36,186 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-08 18:53:36,227 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-08 18:53:36,261 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/java.io.tmpdir/Jetty_localhost_41973_hdfs____lybftu/webapp 2023-03-08 18:53:36,402 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41973 2023-03-08 18:53:36,450 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-08 18:53:36,450 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-08 18:53:36,874 WARN [Listener at localhost/35703] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-08 18:53:37,002 WARN [Listener at localhost/35703] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-08 18:53:37,038 WARN [Listener at localhost/35703] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-08 18:53:37,047 INFO [Listener at localhost/35703] log.Slf4jLog(67): jetty-6.1.26 2023-03-08 18:53:37,053 INFO [Listener at localhost/35703] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/java.io.tmpdir/Jetty_localhost_38329_datanode____8n9dp8/webapp 2023-03-08 18:53:37,179 INFO [Listener at localhost/35703] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38329 2023-03-08 18:53:37,526 WARN [Listener at localhost/36281] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-08 18:53:38,126 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd55d9a3286ef823d: Processing first storage report for DS-e94477ae-343b-4e10-a13e-173abb79b3a5 from datanode d2e0e9f3-5706-4352-ae84-a5bb1a722721 2023-03-08 18:53:38,128 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd55d9a3286ef823d: from storage DS-e94477ae-343b-4e10-a13e-173abb79b3a5 node DatanodeRegistration(127.0.0.1:35845, datanodeUuid=d2e0e9f3-5706-4352-ae84-a5bb1a722721, infoPort=38529, infoSecurePort=0, ipcPort=36281, storageInfo=lv=-57;cid=testClusterID;nsid=386963328;c=1678301615763), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-08 18:53:38,128 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd55d9a3286ef823d: Processing first storage report for DS-33b4e923-ddc3-4279-a4b2-6fa36b6c067a from datanode d2e0e9f3-5706-4352-ae84-a5bb1a722721 2023-03-08 18:53:38,129 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd55d9a3286ef823d: from storage DS-33b4e923-ddc3-4279-a4b2-6fa36b6c067a node DatanodeRegistration(127.0.0.1:35845, datanodeUuid=d2e0e9f3-5706-4352-ae84-a5bb1a722721, infoPort=38529, infoSecurePort=0, ipcPort=36281, storageInfo=lv=-57;cid=testClusterID;nsid=386963328;c=1678301615763), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-08 18:53:38,227 DEBUG [Listener at localhost/36281] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac 2023-03-08 18:53:38,300 INFO [Listener at localhost/36281] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/cluster_8eed2209-15fd-42d6-1e88-8ecffcfc5d58/zookeeper_0, clientPort=54098, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/cluster_8eed2209-15fd-42d6-1e88-8ecffcfc5d58/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/cluster_8eed2209-15fd-42d6-1e88-8ecffcfc5d58/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-08 18:53:38,317 INFO [Listener at localhost/36281] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=54098 2023-03-08 18:53:38,328 INFO [Listener at localhost/36281] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:53:38,330 INFO [Listener at localhost/36281] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:53:39,002 INFO [Listener at localhost/36281] util.FSUtils(471): Created version file at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709 with version=8 2023-03-08 18:53:39,003 INFO [Listener at localhost/36281] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/hbase-staging 2023-03-08 18:53:39,313 INFO [Listener at localhost/36281] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-08 18:53:39,765 INFO [Listener at localhost/36281] client.ConnectionUtils(127): master/jenkins-hbase6:0 server-side Connection retries=3 2023-03-08 18:53:39,798 INFO [Listener at localhost/36281] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 18:53:39,798 INFO [Listener at localhost/36281] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-08 18:53:39,799 INFO [Listener at localhost/36281] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-08 18:53:39,799 INFO [Listener at localhost/36281] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 18:53:39,799 INFO [Listener at localhost/36281] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-08 18:53:39,947 INFO [Listener at localhost/36281] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-08 18:53:40,026 DEBUG [Listener at localhost/36281] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-08 18:53:40,137 INFO [Listener at localhost/36281] ipc.NettyRpcServer(120): Bind to /172.31.2.20:44349 2023-03-08 18:53:40,149 INFO [Listener at localhost/36281] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:53:40,151 INFO [Listener at localhost/36281] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:53:40,174 INFO [Listener at localhost/36281] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44349 connecting to ZooKeeper ensemble=127.0.0.1:54098 2023-03-08 18:53:40,680 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:443490x0, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-08 18:53:40,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:44349-0x101d3e794b30000 connected 2023-03-08 18:53:41,536 DEBUG [Listener at localhost/36281] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 18:53:41,537 DEBUG [Listener at localhost/36281] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 18:53:41,540 DEBUG [Listener at localhost/36281] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-08 18:53:41,548 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44349 2023-03-08 18:53:41,548 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44349 2023-03-08 18:53:41,549 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44349 2023-03-08 18:53:41,549 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44349 2023-03-08 18:53:41,550 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44349 2023-03-08 18:53:41,557 INFO [Listener at localhost/36281] master.HMaster(444): hbase.rootdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709, hbase.cluster.distributed=false 2023-03-08 18:53:41,628 INFO [Listener at localhost/36281] client.ConnectionUtils(127): regionserver/jenkins-hbase6:0 server-side Connection retries=3 2023-03-08 18:53:41,628 INFO [Listener at localhost/36281] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 18:53:41,628 INFO [Listener at localhost/36281] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-08 18:53:41,629 INFO [Listener at localhost/36281] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-08 18:53:41,629 INFO [Listener at localhost/36281] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 18:53:41,629 INFO [Listener at localhost/36281] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-08 18:53:41,634 INFO [Listener at localhost/36281] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-08 18:53:41,637 INFO [Listener at localhost/36281] ipc.NettyRpcServer(120): Bind to /172.31.2.20:46397 2023-03-08 18:53:41,639 INFO [Listener at localhost/36281] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-08 18:53:41,645 DEBUG [Listener at localhost/36281] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-08 18:53:41,646 INFO [Listener at localhost/36281] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:53:41,649 INFO [Listener at localhost/36281] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:53:41,651 INFO [Listener at localhost/36281] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46397 connecting to ZooKeeper ensemble=127.0.0.1:54098 2023-03-08 18:53:41,773 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:463970x0, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-08 18:53:41,774 DEBUG [Listener at localhost/36281] zookeeper.ZKUtil(164): regionserver:463970x0, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 18:53:41,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46397-0x101d3e794b30001 connected 2023-03-08 18:53:41,776 DEBUG [Listener at localhost/36281] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 18:53:41,777 DEBUG [Listener at localhost/36281] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-08 18:53:41,777 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46397 2023-03-08 18:53:41,778 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46397 2023-03-08 18:53:41,778 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46397 2023-03-08 18:53:41,782 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46397 2023-03-08 18:53:41,783 DEBUG [Listener at localhost/36281] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46397 2023-03-08 18:53:41,785 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:53:41,873 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-08 18:53:41,875 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:53:41,988 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-08 18:53:41,988 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-08 18:53:41,988 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:53:41,989 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-08 18:53:41,990 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase6.apache.org,44349,1678301619149 from backup master directory 2023-03-08 18:53:41,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-08 18:53:42,072 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:53:42,073 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-08 18:53:42,073 WARN [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-08 18:53:42,074 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:53:42,077 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-08 18:53:42,078 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-08 18:53:42,162 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/hbase.id with ID: 7027113c-a870-46f8-b906-4bb377d013f9 2023-03-08 18:53:42,209 INFO [master/jenkins-hbase6:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:53:42,227 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:53:42,265 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4905b6f3 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:53:42,300 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@387b40e0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:53:42,325 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-08 18:53:42,327 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-08 18:53:42,347 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-08 18:53:42,348 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-08 18:53:42,350 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-08 18:53:42,354 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-08 18:53:42,356 INFO [master/jenkins-hbase6:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 18:53:42,392 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store-tmp 2023-03-08 18:53:42,422 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:42,422 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-08 18:53:42,423 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 18:53:42,423 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 18:53:42,423 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-08 18:53:42,423 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 18:53:42,423 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 18:53:42,423 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 18:53:42,425 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/WALs/jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:53:42,450 INFO [master/jenkins-hbase6:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase6.apache.org%2C44349%2C1678301619149, suffix=, logDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/WALs/jenkins-hbase6.apache.org,44349,1678301619149, archiveDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/oldWALs, maxLogs=10 2023-03-08 18:53:42,515 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35845,DS-e94477ae-343b-4e10-a13e-173abb79b3a5,DISK] 2023-03-08 18:53:42,524 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-08 18:53:42,586 INFO [master/jenkins-hbase6:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/WALs/jenkins-hbase6.apache.org,44349,1678301619149/jenkins-hbase6.apache.org%2C44349%2C1678301619149.1678301622465 2023-03-08 18:53:42,587 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35845,DS-e94477ae-343b-4e10-a13e-173abb79b3a5,DISK]] 2023-03-08 18:53:42,588 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:53:42,588 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:42,592 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 18:53:42,593 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 18:53:42,657 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-08 18:53:42,664 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-08 18:53:42,690 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-08 18:53:42,703 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:42,711 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-08 18:53:42,715 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-08 18:53:42,734 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 18:53:42,740 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:53:42,741 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75197694, jitterRate=0.12053295969963074}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:53:42,741 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 18:53:42,744 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-08 18:53:42,766 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-08 18:53:42,766 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-08 18:53:42,768 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-08 18:53:42,771 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-08 18:53:42,817 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 46 msec 2023-03-08 18:53:42,818 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-08 18:53:42,845 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-08 18:53:42,852 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-08 18:53:42,884 INFO [master/jenkins-hbase6:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-08 18:53:42,890 INFO [master/jenkins-hbase6:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-08 18:53:42,892 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-08 18:53:42,899 INFO [master/jenkins-hbase6:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-08 18:53:42,905 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-08 18:53:42,914 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:53:42,916 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-08 18:53:42,917 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-08 18:53:42,932 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-08 18:53:42,937 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-08 18:53:42,937 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-08 18:53:42,937 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:53:42,938 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase6.apache.org,44349,1678301619149, sessionid=0x101d3e794b30000, setting cluster-up flag (Was=false) 2023-03-08 18:53:42,952 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:53:42,959 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-08 18:53:42,960 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:53:42,966 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:53:42,972 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-08 18:53:42,973 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:53:42,976 WARN [master/jenkins-hbase6:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp 2023-03-08 18:53:42,987 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(952): ClusterId : 7027113c-a870-46f8-b906-4bb377d013f9 2023-03-08 18:53:42,993 DEBUG [RS:0;jenkins-hbase6:46397] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-08 18:53:43,000 DEBUG [RS:0;jenkins-hbase6:46397] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-08 18:53:43,001 DEBUG [RS:0;jenkins-hbase6:46397] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-08 18:53:43,004 DEBUG [RS:0;jenkins-hbase6:46397] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-08 18:53:43,005 DEBUG [RS:0;jenkins-hbase6:46397] zookeeper.ReadOnlyZKClient(139): Connect 0x30bc36e3 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:53:43,015 DEBUG [RS:0;jenkins-hbase6:46397] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3e7bce84, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:53:43,015 DEBUG [RS:0;jenkins-hbase6:46397] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a445d93, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase6.apache.org/172.31.2.20:0 2023-03-08 18:53:43,043 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase6:46397 2023-03-08 18:53:43,047 INFO [RS:0;jenkins-hbase6:46397] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-08 18:53:43,047 INFO [RS:0;jenkins-hbase6:46397] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-08 18:53:43,047 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1023): About to register with Master. 2023-03-08 18:53:43,050 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase6.apache.org,44349,1678301619149 with isa=jenkins-hbase6.apache.org/172.31.2.20:46397, startcode=1678301621627 2023-03-08 18:53:43,068 DEBUG [RS:0;jenkins-hbase6:46397] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-08 18:53:43,123 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-08 18:53:43,134 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:54407, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-08 18:53:43,136 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase6:0, corePoolSize=5, maxPoolSize=5 2023-03-08 18:53:43,136 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase6:0, corePoolSize=5, maxPoolSize=5 2023-03-08 18:53:43,136 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=5, maxPoolSize=5 2023-03-08 18:53:43,136 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=5, maxPoolSize=5 2023-03-08 18:53:43,136 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase6:0, corePoolSize=10, maxPoolSize=10 2023-03-08 18:53:43,137 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,137 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=2, maxPoolSize=2 2023-03-08 18:53:43,137 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,142 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1678301653142 2023-03-08 18:53:43,144 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-08 18:53:43,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-03-08 18:53:43,150 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-08 18:53:43,150 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-08 18:53:43,156 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-08 18:53:43,156 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-08 18:53:43,163 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-08 18:53:43,163 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-08 18:53:43,164 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-08 18:53:43,164 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-08 18:53:43,167 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,169 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-08 18:53:43,171 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-08 18:53:43,171 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-08 18:53:43,174 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-08 18:53:43,174 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-08 18:53:43,176 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.large.0-1678301623176,5,FailOnTimeoutGroup] 2023-03-08 18:53:43,177 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.small.0-1678301623177,5,FailOnTimeoutGroup] 2023-03-08 18:53:43,177 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,178 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-08 18:53:43,179 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,179 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,181 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(2829): Master is not running yet 2023-03-08 18:53:43,182 WARN [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-03-08 18:53:43,283 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase6.apache.org,44349,1678301619149 with isa=jenkins-hbase6.apache.org/172.31.2.20:46397, startcode=1678301621627 2023-03-08 18:53:43,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.ServerManager(394): Registering regionserver=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:43,299 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709 2023-03-08 18:53:43,299 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:35703 2023-03-08 18:53:43,299 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-08 18:53:43,304 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 18:53:43,305 DEBUG [RS:0;jenkins-hbase6:46397] zookeeper.ZKUtil(162): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:43,305 WARN [RS:0;jenkins-hbase6:46397] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-08 18:53:43,306 INFO [RS:0;jenkins-hbase6:46397] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 18:53:43,306 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1947): logDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/WALs/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:43,307 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase6.apache.org,46397,1678301621627] 2023-03-08 18:53:43,316 DEBUG [RS:0;jenkins-hbase6:46397] zookeeper.ZKUtil(162): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:43,330 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-08 18:53:43,340 INFO [RS:0;jenkins-hbase6:46397] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-08 18:53:43,365 INFO [RS:0;jenkins-hbase6:46397] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-08 18:53:43,369 INFO [RS:0;jenkins-hbase6:46397] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:53:43,369 INFO [RS:0;jenkins-hbase6:46397] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,371 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-08 18:53:43,378 INFO [RS:0;jenkins-hbase6:46397] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,379 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,379 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,379 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,380 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,380 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,380 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase6:0, corePoolSize=2, maxPoolSize=2 2023-03-08 18:53:43,380 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,380 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,380 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,381 DEBUG [RS:0;jenkins-hbase6:46397] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-03-08 18:53:43,382 INFO [RS:0;jenkins-hbase6:46397] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,382 INFO [RS:0;jenkins-hbase6:46397] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,382 INFO [RS:0;jenkins-hbase6:46397] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,401 INFO [RS:0;jenkins-hbase6:46397] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-08 18:53:43,404 INFO [RS:0;jenkins-hbase6:46397] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,46397,1678301621627-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:43,424 INFO [RS:0;jenkins-hbase6:46397] regionserver.Replication(203): jenkins-hbase6.apache.org,46397,1678301621627 started 2023-03-08 18:53:43,424 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1638): Serving as jenkins-hbase6.apache.org,46397,1678301621627, RpcServer on jenkins-hbase6.apache.org/172.31.2.20:46397, sessionid=0x101d3e794b30001 2023-03-08 18:53:43,424 DEBUG [RS:0;jenkins-hbase6:46397] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-08 18:53:43,425 DEBUG [RS:0;jenkins-hbase6:46397] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:43,425 DEBUG [RS:0;jenkins-hbase6:46397] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase6.apache.org,46397,1678301621627' 2023-03-08 18:53:43,425 DEBUG [RS:0;jenkins-hbase6:46397] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:43,426 DEBUG [RS:0;jenkins-hbase6:46397] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:43,426 DEBUG [RS:0;jenkins-hbase6:46397] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-08 18:53:43,426 DEBUG [RS:0;jenkins-hbase6:46397] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-08 18:53:43,426 DEBUG [RS:0;jenkins-hbase6:46397] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:43,427 DEBUG [RS:0;jenkins-hbase6:46397] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase6.apache.org,46397,1678301621627' 2023-03-08 18:53:43,427 DEBUG [RS:0;jenkins-hbase6:46397] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 18:53:43,427 DEBUG [RS:0;jenkins-hbase6:46397] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 18:53:43,428 DEBUG [RS:0;jenkins-hbase6:46397] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-08 18:53:43,428 INFO [RS:0;jenkins-hbase6:46397] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-08 18:53:43,428 INFO [RS:0;jenkins-hbase6:46397] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-08 18:53:43,543 INFO [RS:0;jenkins-hbase6:46397] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase6.apache.org%2C46397%2C1678301621627, suffix=, logDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/WALs/jenkins-hbase6.apache.org,46397,1678301621627, archiveDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/oldWALs, maxLogs=32 2023-03-08 18:53:43,566 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35845,DS-e94477ae-343b-4e10-a13e-173abb79b3a5,DISK] 2023-03-08 18:53:43,572 INFO [RS:0;jenkins-hbase6:46397] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/WALs/jenkins-hbase6.apache.org,46397,1678301621627/jenkins-hbase6.apache.org%2C46397%2C1678301621627.1678301623546 2023-03-08 18:53:43,572 DEBUG [RS:0;jenkins-hbase6:46397] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35845,DS-e94477ae-343b-4e10-a13e-173abb79b3a5,DISK]] 2023-03-08 18:53:43,595 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-08 18:53:43,596 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-08 18:53:43,597 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709 2023-03-08 18:53:44,012 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:44,015 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-08 18:53:44,018 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/info 2023-03-08 18:53:44,019 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-08 18:53:44,020 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:44,020 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-08 18:53:44,023 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/rep_barrier 2023-03-08 18:53:44,024 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-08 18:53:44,024 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:44,025 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-08 18:53:44,027 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/table 2023-03-08 18:53:44,028 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-08 18:53:44,029 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:44,030 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740 2023-03-08 18:53:44,031 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740 2023-03-08 18:53:44,036 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-08 18:53:44,039 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-08 18:53:44,043 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:53:44,045 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59049035, jitterRate=-0.12010081112384796}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-08 18:53:44,045 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-08 18:53:44,045 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-08 18:53:44,045 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-08 18:53:44,045 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-08 18:53:44,045 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-08 18:53:44,046 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-08 18:53:44,047 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-08 18:53:44,047 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-08 18:53:44,052 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-08 18:53:44,053 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-08 18:53:44,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-08 18:53:44,077 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-08 18:53:44,080 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-08 18:53:44,232 DEBUG [jenkins-hbase6:44349] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-08 18:53:44,236 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase6.apache.org,46397,1678301621627, state=OPENING 2023-03-08 18:53:44,243 DEBUG [PEWorker-2] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-08 18:53:44,244 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:53:44,245 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-08 18:53:44,248 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:53:44,434 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:44,436 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 18:53:44,440 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:58014, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 18:53:44,450 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-08 18:53:44,451 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 18:53:44,454 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase6.apache.org%2C46397%2C1678301621627.meta, suffix=.meta, logDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/WALs/jenkins-hbase6.apache.org,46397,1678301621627, archiveDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/oldWALs, maxLogs=32 2023-03-08 18:53:44,470 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35845,DS-e94477ae-343b-4e10-a13e-173abb79b3a5,DISK] 2023-03-08 18:53:44,475 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/WALs/jenkins-hbase6.apache.org,46397,1678301621627/jenkins-hbase6.apache.org%2C46397%2C1678301621627.meta.1678301624455.meta 2023-03-08 18:53:44,475 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35845,DS-e94477ae-343b-4e10-a13e-173abb79b3a5,DISK]] 2023-03-08 18:53:44,475 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:53:44,477 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-08 18:53:44,496 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-08 18:53:44,503 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-08 18:53:44,510 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-08 18:53:44,511 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:44,511 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-08 18:53:44,511 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-08 18:53:44,514 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-08 18:53:44,516 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/info 2023-03-08 18:53:44,516 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/info 2023-03-08 18:53:44,517 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-08 18:53:44,518 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:44,518 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-08 18:53:44,519 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/rep_barrier 2023-03-08 18:53:44,519 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/rep_barrier 2023-03-08 18:53:44,520 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-08 18:53:44,521 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:44,521 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-08 18:53:44,524 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/table 2023-03-08 18:53:44,524 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/table 2023-03-08 18:53:44,525 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-08 18:53:44,526 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:44,527 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740 2023-03-08 18:53:44,530 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740 2023-03-08 18:53:44,535 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-08 18:53:44,538 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-08 18:53:44,539 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66863335, jitterRate=-0.003658667206764221}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-08 18:53:44,539 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-08 18:53:44,549 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1678301624427 2023-03-08 18:53:44,565 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-08 18:53:44,566 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-08 18:53:44,566 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase6.apache.org,46397,1678301621627, state=OPEN 2023-03-08 18:53:44,569 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-08 18:53:44,569 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-08 18:53:44,573 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-08 18:53:44,573 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase6.apache.org,46397,1678301621627 in 321 msec 2023-03-08 18:53:44,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-08 18:53:44,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 513 msec 2023-03-08 18:53:44,583 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5440 sec 2023-03-08 18:53:44,583 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1678301624583, completionTime=-1 2023-03-08 18:53:44,583 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-08 18:53:44,584 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-03-08 18:53:44,647 DEBUG [hconnection-0x3b969aae-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:53:44,651 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:58016, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:53:44,668 INFO [master/jenkins-hbase6:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-03-08 18:53:44,668 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1678301684668 2023-03-08 18:53:44,668 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1678301744668 2023-03-08 18:53:44,668 INFO [master/jenkins-hbase6:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 84 msec 2023-03-08 18:53:44,692 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44349,1678301619149-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:44,693 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44349,1678301619149-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:44,693 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44349,1678301619149-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:44,695 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase6:44349, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:44,695 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-08 18:53:44,702 DEBUG [master/jenkins-hbase6:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-08 18:53:44,713 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-08 18:53:44,715 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-08 18:53:44,727 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-08 18:53:44,730 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:53:44,733 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:53:44,761 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:44,763 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad empty. 2023-03-08 18:53:44,764 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:44,764 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-08 18:53:44,787 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-08 18:53:44,789 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => bd9899deeb96650165ed7c0a6cf10aad, NAME => 'hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:53:45,202 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:45,202 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing bd9899deeb96650165ed7c0a6cf10aad, disabling compactions & flushes 2023-03-08 18:53:45,202 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:53:45,202 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:53:45,203 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. after waiting 0 ms 2023-03-08 18:53:45,203 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:53:45,203 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:53:45,203 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for bd9899deeb96650165ed7c0a6cf10aad: 2023-03-08 18:53:45,208 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:53:45,224 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678301625211"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301625211"}]},"ts":"1678301625211"} 2023-03-08 18:53:45,252 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:53:45,254 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:53:45,259 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301625255"}]},"ts":"1678301625255"} 2023-03-08 18:53:45,264 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-08 18:53:45,276 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=bd9899deeb96650165ed7c0a6cf10aad, ASSIGN}] 2023-03-08 18:53:45,278 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=bd9899deeb96650165ed7c0a6cf10aad, ASSIGN 2023-03-08 18:53:45,280 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=bd9899deeb96650165ed7c0a6cf10aad, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:53:45,431 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=bd9899deeb96650165ed7c0a6cf10aad, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:45,431 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678301625431"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301625431"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301625431"}]},"ts":"1678301625431"} 2023-03-08 18:53:45,436 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure bd9899deeb96650165ed7c0a6cf10aad, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:53:45,596 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:53:45,596 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bd9899deeb96650165ed7c0a6cf10aad, NAME => 'hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:53:45,597 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:45,597 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:45,597 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:45,597 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:45,600 INFO [StoreOpener-bd9899deeb96650165ed7c0a6cf10aad-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:45,602 DEBUG [StoreOpener-bd9899deeb96650165ed7c0a6cf10aad-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad/info 2023-03-08 18:53:45,602 DEBUG [StoreOpener-bd9899deeb96650165ed7c0a6cf10aad-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad/info 2023-03-08 18:53:45,602 INFO [StoreOpener-bd9899deeb96650165ed7c0a6cf10aad-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bd9899deeb96650165ed7c0a6cf10aad columnFamilyName info 2023-03-08 18:53:45,603 INFO [StoreOpener-bd9899deeb96650165ed7c0a6cf10aad-1] regionserver.HStore(310): Store=bd9899deeb96650165ed7c0a6cf10aad/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:45,605 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:45,606 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:45,610 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:53:45,614 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:53:45,615 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened bd9899deeb96650165ed7c0a6cf10aad; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67421025, jitterRate=0.004651561379432678}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:53:45,615 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for bd9899deeb96650165ed7c0a6cf10aad: 2023-03-08 18:53:45,617 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad., pid=6, masterSystemTime=1678301625589 2023-03-08 18:53:45,621 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:53:45,621 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:53:45,623 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=bd9899deeb96650165ed7c0a6cf10aad, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:45,623 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678301625622"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301625622"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301625622"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301625622"}]},"ts":"1678301625622"} 2023-03-08 18:53:45,630 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-08 18:53:45,630 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure bd9899deeb96650165ed7c0a6cf10aad, server=jenkins-hbase6.apache.org,46397,1678301621627 in 191 msec 2023-03-08 18:53:45,634 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-08 18:53:45,634 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=bd9899deeb96650165ed7c0a6cf10aad, ASSIGN in 354 msec 2023-03-08 18:53:45,635 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:53:45,636 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301625636"}]},"ts":"1678301625636"} 2023-03-08 18:53:45,639 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-08 18:53:45,642 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:53:45,645 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 926 msec 2023-03-08 18:53:45,730 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-08 18:53:45,733 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-08 18:53:45,733 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:53:45,780 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-08 18:53:45,797 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 18:53:45,804 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 36 msec 2023-03-08 18:53:45,813 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-08 18:53:45,825 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 18:53:45,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-03-08 18:53:45,840 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-08 18:53:45,843 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-08 18:53:45,843 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.769sec 2023-03-08 18:53:45,845 INFO [master/jenkins-hbase6:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-08 18:53:45,847 INFO [master/jenkins-hbase6:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-08 18:53:45,847 INFO [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-08 18:53:45,848 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44349,1678301619149-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-08 18:53:45,850 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44349,1678301619149-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-08 18:53:45,863 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-08 18:53:45,894 DEBUG [Listener at localhost/36281] zookeeper.ReadOnlyZKClient(139): Connect 0x0672b468 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:53:45,899 DEBUG [Listener at localhost/36281] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@561bf69b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:53:45,911 DEBUG [hconnection-0x78f916d2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:53:45,922 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:58022, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:53:45,932 INFO [Listener at localhost/36281] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:53:45,933 DEBUG [Listener at localhost/36281] zookeeper.ReadOnlyZKClient(139): Connect 0x33fc1e18 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:53:45,938 DEBUG [Listener at localhost/36281] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b586bb5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:53:45,939 INFO [Listener at localhost/36281] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:53:45,961 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=184, AvailableMemoryMB=7393 2023-03-08 18:53:45,972 DEBUG [Listener at localhost/36281] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:53:45,976 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:58696, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:53:45,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 18:53:45,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:45,992 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:53:45,994 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:53:45,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-08 18:53:45,997 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:45,999 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32 empty. 2023-03-08 18:53:46,000 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:46,000 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-08 18:53:46,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 18:53:46,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 18:53:46,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 18:53:46,420 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-08 18:53:46,423 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8cccb8903f14f2a6b0b76a282ea1eb32, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:53:46,438 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:46,439 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 8cccb8903f14f2a6b0b76a282ea1eb32, disabling compactions & flushes 2023-03-08 18:53:46,439 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:46,439 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:46,439 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. after waiting 0 ms 2023-03-08 18:53:46,439 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:46,439 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:46,439 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:46,444 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:53:46,447 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678301626447"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301626447"}]},"ts":"1678301626447"} 2023-03-08 18:53:46,451 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:53:46,453 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:53:46,453 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301626453"}]},"ts":"1678301626453"} 2023-03-08 18:53:46,455 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-08 18:53:46,460 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8cccb8903f14f2a6b0b76a282ea1eb32, ASSIGN}] 2023-03-08 18:53:46,462 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8cccb8903f14f2a6b0b76a282ea1eb32, ASSIGN 2023-03-08 18:53:46,464 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8cccb8903f14f2a6b0b76a282ea1eb32, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:53:46,615 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=8cccb8903f14f2a6b0b76a282ea1eb32, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:46,615 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678301626615"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301626615"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301626615"}]},"ts":"1678301626615"} 2023-03-08 18:53:46,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 18:53:46,618 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 8cccb8903f14f2a6b0b76a282ea1eb32, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:53:46,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:46,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8cccb8903f14f2a6b0b76a282ea1eb32, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:53:46,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:46,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:46,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:46,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:46,784 INFO [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:46,787 DEBUG [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:46,787 DEBUG [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:46,790 INFO [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8cccb8903f14f2a6b0b76a282ea1eb32 columnFamilyName family1 2023-03-08 18:53:46,794 INFO [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] regionserver.HStore(310): Store=8cccb8903f14f2a6b0b76a282ea1eb32/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:46,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:46,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:46,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:46,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:53:46,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 8cccb8903f14f2a6b0b76a282ea1eb32; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67221359, jitterRate=0.001676306128501892}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:53:46,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:46,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32., pid=11, masterSystemTime=1678301626771 2023-03-08 18:53:46,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:46,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:46,816 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=8cccb8903f14f2a6b0b76a282ea1eb32, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:46,817 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678301626816"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301626816"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301626816"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301626816"}]},"ts":"1678301626816"} 2023-03-08 18:53:46,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-08 18:53:46,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 8cccb8903f14f2a6b0b76a282ea1eb32, server=jenkins-hbase6.apache.org,46397,1678301621627 in 202 msec 2023-03-08 18:53:46,827 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-08 18:53:46,827 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8cccb8903f14f2a6b0b76a282ea1eb32, ASSIGN in 363 msec 2023-03-08 18:53:46,828 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:53:46,829 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301626828"}]},"ts":"1678301626828"} 2023-03-08 18:53:46,831 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-08 18:53:46,834 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:53:46,836 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 845 msec 2023-03-08 18:53:47,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 18:53:47,120 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-08 18:53:47,143 DEBUG [hconnection-0x206d21f7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:53:47,147 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:58024, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:53:47,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:47,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:47,314 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:47,314 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:47,315 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:47,315 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:47,315 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:47,315 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:47,317 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:47,317 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:47,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:47,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:47,318 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:47,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:47,318 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:47,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:47,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:47,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:47,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:47,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:47,321 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:47,321 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:47,322 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:47,322 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:47,322 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:47,323 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:47,323 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:47,324 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 18:53:47,897 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130cde836015ea84cba86f44f34a8346775 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130cde836015ea84cba86f44f34a8346775 2023-03-08 18:53:48,313 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/e66a380ac8304766b653c08cd34daa2d 2023-03-08 18:53:48,324 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/e66a380ac8304766b653c08cd34daa2d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e66a380ac8304766b653c08cd34daa2d 2023-03-08 18:53:48,333 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e66a380ac8304766b653c08cd34daa2d, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-08 18:53:48,336 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 1012ms, sequenceid=5, compaction requested=false 2023-03-08 18:53:48,338 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:48,340 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:48,340 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:48,340 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:48,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:48,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:48,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:48,344 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,344 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:48,345 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,345 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:48,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:48,347 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:48,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78c3770e[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:48,347 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,349 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,349 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:48,349 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,349 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:48,349 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:48,349 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:48,352 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,352 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:48,352 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:48,352 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:48,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:48,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:48,356 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:48,356 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:48,356 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:48,356 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,356 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:48,358 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,358 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:48,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,358 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:48,359 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:48,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:48,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:48,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,361 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:48,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:48,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:48,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:48,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:48,369 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:48,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:48,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,370 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,370 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:48,372 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:48,372 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:48,472 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:48,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:48,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:48,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:48,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,491 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:48,491 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:48,492 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:48,492 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:48,493 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,493 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,494 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:48,494 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:48,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:48,495 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:48,495 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:48,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,495 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:48,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,496 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,496 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:48,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:48,496 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:48,496 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:48,497 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:48,497 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:48,497 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:48,497 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:48,497 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:48,497 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 18:53:48,924 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e567a85f279d40bb9cf636892762a31d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128e567a85f279d40bb9cf636892762a31d 2023-03-08 18:53:49,308 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-08 18:53:49,308 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-08 18:53:49,334 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/8317d72e8fec41c98ac2b6478d34472a 2023-03-08 18:53:49,336 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-08 18:53:49,337 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-08 18:53:49,347 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/8317d72e8fec41c98ac2b6478d34472a as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/8317d72e8fec41c98ac2b6478d34472a 2023-03-08 18:53:49,356 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/8317d72e8fec41c98ac2b6478d34472a, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 18:53:49,358 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 861ms, sequenceid=9, compaction requested=false 2023-03-08 18:53:49,358 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:49,358 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:49,358 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:49,359 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:49,359 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,359 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:49,359 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:49,362 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,362 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:49,363 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,363 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:49,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:49,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:49,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:49,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:49,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26124c18[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:49,365 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:49,365 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,367 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,367 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:49,367 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:49,367 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:49,368 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,368 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:49,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,369 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:49,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:49,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:49,370 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:49,370 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:49,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:49,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:49,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:49,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:49,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:49,374 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:49,374 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:49,374 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:49,374 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,374 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:49,384 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,384 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:49,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:49,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,384 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:49,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:49,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:49,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:49,385 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:49,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,388 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:49,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,394 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,394 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,394 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:49,394 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,394 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:49,394 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,395 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:49,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:49,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:49,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:49,395 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:49,395 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,395 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,396 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:49,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:49,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:49,396 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:49,498 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:49,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:49,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:49,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:49,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,516 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:49,516 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:49,516 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:49,517 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:49,518 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,518 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,523 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,523 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:49,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:49,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:49,527 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,527 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:49,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:49,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:49,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,528 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:49,528 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:49,529 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:49,529 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:49,529 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:49,529 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:49,529 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:49,529 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:53:49,591 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512014e363fc553d74137a5ba10ecd59a938d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512014e363fc553d74137a5ba10ecd59a938d 2023-03-08 18:53:49,610 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/73500d03b3dc4f75be8664f1bff19ffd 2023-03-08 18:53:49,619 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/73500d03b3dc4f75be8664f1bff19ffd as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/73500d03b3dc4f75be8664f1bff19ffd 2023-03-08 18:53:49,627 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/73500d03b3dc4f75be8664f1bff19ffd, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 18:53:49,629 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 100ms, sequenceid=13, compaction requested=true 2023-03-08 18:53:49,629 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:49,629 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:49,629 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:49,630 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:49,630 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,630 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:49,630 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:49,633 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,633 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:49,634 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,634 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:49,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:49,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dad958b[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:49,636 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:49,636 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,638 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,638 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:49,638 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:49,638 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:49,638 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,638 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:49,640 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,640 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:49,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:49,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:49,640 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:49,640 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:49,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:49,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:49,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:49,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:49,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:49,645 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:49,645 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:49,645 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:49,645 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,645 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:49,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:49,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:49,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:49,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:49,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:49,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:49,651 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:49,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:49,654 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:49,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:49,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:49,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:49,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:49,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,660 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:49,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:49,661 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:49,661 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:49,661 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,661 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:49,661 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:49,762 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:49,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:49,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,775 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:49,775 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:49,775 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:49,775 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:49,776 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,776 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,777 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,777 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:49,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:49,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:49,780 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:49,780 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:49,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:49,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,786 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:49,786 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:49,786 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:49,786 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:49,786 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:49,787 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:49,787 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:49,788 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:49,788 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:49,788 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:53:49,827 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ab2b8696af054d3a8f3a6c647f4e452a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205ab2b8696af054d3a8f3a6c647f4e452a 2023-03-08 18:53:50,058 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:53:50,246 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/89366c7c71584f128101bf3ee6476bb4 2023-03-08 18:53:50,256 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/89366c7c71584f128101bf3ee6476bb4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/89366c7c71584f128101bf3ee6476bb4 2023-03-08 18:53:50,265 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/89366c7c71584f128101bf3ee6476bb4, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 18:53:50,266 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 478ms, sequenceid=17, compaction requested=true 2023-03-08 18:53:50,267 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:50,267 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:50,267 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:50,267 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:50,267 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,267 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:50,267 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:50,270 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,270 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:50,271 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,271 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:50,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:50,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e1ee5db[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:50,273 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:50,273 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,274 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,275 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:50,275 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:50,275 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,275 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:50,275 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:50,277 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,277 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:50,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:50,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:50,277 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:50,277 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:50,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:50,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:50,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:50,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:50,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:50,282 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:50,282 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:50,282 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:50,282 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,282 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:50,284 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,284 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,284 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:50,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:50,285 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,287 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:50,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,294 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:50,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:50,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:50,294 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:50,295 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:50,295 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:50,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:50,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:50,395 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:50,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:50,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:50,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:50,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,407 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:50,407 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:50,407 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:50,407 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:50,408 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,408 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,410 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:50,410 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:50,410 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,410 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:50,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,411 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:50,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,411 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:50,412 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:50,412 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:50,412 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:50,412 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:50,412 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:50,412 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:50,412 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:53:50,433 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228cc7a7af3fb404ce9a11d67c4b7a8587c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228cc7a7af3fb404ce9a11d67c4b7a8587c 2023-03-08 18:53:50,845 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/ea1103c310e94cb4b1e51b810f6fd1b9 2023-03-08 18:53:50,853 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/ea1103c310e94cb4b1e51b810f6fd1b9 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/ea1103c310e94cb4b1e51b810f6fd1b9 2023-03-08 18:53:50,862 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/ea1103c310e94cb4b1e51b810f6fd1b9, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 18:53:50,863 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 451ms, sequenceid=21, compaction requested=true 2023-03-08 18:53:50,863 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:50,863 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:50,863 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:50,864 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:50,864 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,864 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:50,864 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:50,869 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,869 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:50,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:50,869 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,869 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:50,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:50,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:50,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:50,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:50,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d069a11[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:50,871 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:50,871 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,873 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,873 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:50,873 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:50,873 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:50,873 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,873 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:50,874 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,874 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:50,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:50,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:50,874 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:50,875 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:50,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:50,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:50,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:50,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:50,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:50,878 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:50,878 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:50,878 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:50,878 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,878 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:50,881 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,881 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:50,881 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:50,882 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:50,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:50,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:50,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,883 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,886 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,886 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:50,886 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,886 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:50,886 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:50,886 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:50,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 18:53:50,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:50,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:50,886 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:50,886 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:50,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:50,887 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:50,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:50,888 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:50,988 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:50,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:50,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:51,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:51,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,002 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:51,002 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:51,002 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:51,002 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:51,003 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,003 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,004 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:51,004 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:51,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:51,005 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,005 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:51,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:51,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,006 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:51,007 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:51,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:51,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:51,008 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:51,008 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:51,008 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:51,008 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:51,008 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:53:51,050 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231db9179a0c667499783b85d46dcee3fd3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231db9179a0c667499783b85d46dcee3fd3 2023-03-08 18:53:51,064 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/88f7dfc78d3c430c9d661d5cd577d78b 2023-03-08 18:53:51,075 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/88f7dfc78d3c430c9d661d5cd577d78b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/88f7dfc78d3c430c9d661d5cd577d78b 2023-03-08 18:53:51,084 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/88f7dfc78d3c430c9d661d5cd577d78b, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 18:53:51,086 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 78ms, sequenceid=25, compaction requested=true 2023-03-08 18:53:51,086 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:51,086 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:51,086 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:51,087 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:51,087 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,087 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:51,087 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:51,089 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,089 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:51,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:51,090 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,090 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:51,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:51,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:51,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49406b25[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:51,102 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:51,103 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,104 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,104 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:51,104 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:51,104 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:51,105 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,105 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:51,107 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:51,108 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:51,108 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:51,108 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:51,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:51,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:51,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:51,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:51,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:51,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:51,119 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:51,119 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:51,119 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:51,119 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,119 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:51,122 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,122 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,122 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:51,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:51,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:51,123 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:51,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,128 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:51,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,137 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:51,138 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:51,137 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:51,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:51,138 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:51,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:51,138 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:51,138 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,139 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,139 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,139 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,139 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,139 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:51,139 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:51,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:51,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:51,239 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:51,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:51,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:51,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:51,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,251 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:51,251 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:51,252 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:51,252 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:51,252 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,252 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,254 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:51,254 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:51,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:51,254 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,254 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:51,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:51,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:51,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,256 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:51,256 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:51,256 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:51,258 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:51,258 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:51,258 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:51,258 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:51,258 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:53:51,680 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101b36e346a283f4b1a9474ed10065aceb1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b36e346a283f4b1a9474ed10065aceb1 2023-03-08 18:53:51,710 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/e639902ff47a4b549f8ebfde6fb693c8 2023-03-08 18:53:51,718 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/e639902ff47a4b549f8ebfde6fb693c8 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e639902ff47a4b549f8ebfde6fb693c8 2023-03-08 18:53:51,727 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e639902ff47a4b549f8ebfde6fb693c8, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 18:53:51,728 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 470ms, sequenceid=29, compaction requested=true 2023-03-08 18:53:51,728 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:51,729 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:51,729 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:51,729 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:51,729 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,729 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:51,729 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:51,731 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,731 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:51,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:51,732 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,732 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:51,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:51,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:51,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:51,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:51,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5850ae4b[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:51,734 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:51,734 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,737 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:51,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:51,737 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,737 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:51,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:51,739 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:51,739 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,739 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:51,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:51,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:51,739 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:51,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:51,742 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:51,742 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:51,742 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:51,742 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,742 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:51,744 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,744 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:51,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:51,744 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:51,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:51,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:51,745 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:51,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:51,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,746 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,753 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,753 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,753 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:51,753 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:51,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:51,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:51,753 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:51,753 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:51,753 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,754 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,754 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,754 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,754 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:51,754 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:51,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:51,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:51,854 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:51,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:51,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:51,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:51,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,866 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:51,866 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:51,866 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:51,866 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:51,867 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,867 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,868 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:51,868 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:51,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:51,869 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:51,869 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:51,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:51,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,870 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,870 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:51,870 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:51,870 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:51,870 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:51,871 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:51,871 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:51,871 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:51,871 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:51,871 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:51,871 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:53:52,294 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010323a8c14054634e849b8299aa59cad29f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010323a8c14054634e849b8299aa59cad29f 2023-03-08 18:53:52,705 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/70ba943aa9fd41b39c910889995d919e 2023-03-08 18:53:52,714 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/70ba943aa9fd41b39c910889995d919e as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/70ba943aa9fd41b39c910889995d919e 2023-03-08 18:53:52,723 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/70ba943aa9fd41b39c910889995d919e, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 18:53:52,724 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 853ms, sequenceid=33, compaction requested=true 2023-03-08 18:53:52,724 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:52,724 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:52,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:52,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:52,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:52,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:52,727 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,727 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:52,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:52,727 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:52,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:52,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:52,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:52,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:52,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3189a8a7[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:52,729 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:52,729 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,730 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,731 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:52,731 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:52,731 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:52,731 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,731 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:52,732 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,732 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:52,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:52,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:52,732 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:52,733 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:52,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:52,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:52,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:52,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:52,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:52,736 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:52,736 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:52,736 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:52,736 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,736 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:52,739 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,739 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,740 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:52,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:52,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:52,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:52,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:52,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:52,740 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:52,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,742 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:52,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:52,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 18:53:52,745 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:52,746 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:52,745 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:52,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:52,746 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:52,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:52,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:52,847 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:52,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:52,858 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:52,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:52,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,861 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:52,861 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:52,862 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:52,862 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:52,862 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,862 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,864 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:52,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:52,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:52,864 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:52,864 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:52,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,865 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:52,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,865 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,866 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:52,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:52,867 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:52,867 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:52,868 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:52,868 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:52,868 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:52,868 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:52,868 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:52,868 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:53:53,296 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509076bc7a19d8a834b36955378cb81b29f56 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509076bc7a19d8a834b36955378cb81b29f56 2023-03-08 18:53:53,707 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/c97a37cb7b7a46b8be89f0817f015fe7 2023-03-08 18:53:53,718 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/c97a37cb7b7a46b8be89f0817f015fe7 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/c97a37cb7b7a46b8be89f0817f015fe7 2023-03-08 18:53:53,726 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/c97a37cb7b7a46b8be89f0817f015fe7, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 18:53:53,727 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 859ms, sequenceid=37, compaction requested=true 2023-03-08 18:53:53,727 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:53,727 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:53,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:53,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:53,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:53,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:53,730 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,730 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:53,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:53,731 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,731 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:53,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:53,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:53,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:53,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:53,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78167a95[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:53,733 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:53,733 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,735 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,735 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:53,735 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:53,735 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:53,735 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,735 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:53,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:53,737 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:53,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:53,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:53,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:53,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:53,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:53,742 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:53,742 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:53,742 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:53,742 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,742 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:53,744 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,744 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:53,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:53,744 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:53,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:53,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:53,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:53,745 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:53,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,747 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:53,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,749 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,750 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,750 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:53,750 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:53,750 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:53,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 18:53:53,750 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:53,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:53,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:53,751 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,751 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,751 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,751 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,752 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:53,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:53,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:53,752 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:53,852 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:53,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:53,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:53:53,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 18:53:53,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,890 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:53,890 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:53,890 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 18:53:53,890 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 18:53:53,891 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,891 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,893 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,893 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:53,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:53,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:53,893 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:53,893 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:53:53,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:53:53,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 18:53:53,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:53,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 18:53:53,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:53:53,895 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 18:53:53,896 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:53:53,896 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:53:53,896 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:53,896 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. started... 2023-03-08 18:53:53,896 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cccb8903f14f2a6b0b76a282ea1eb32 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:53:53,934 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a99f1094860e4690b76efc5bdc00d4d6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a99f1094860e4690b76efc5bdc00d4d6 2023-03-08 18:53:54,347 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/466b6af109154fa5afbc6008d6a81984 2023-03-08 18:53:54,355 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/466b6af109154fa5afbc6008d6a81984 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/466b6af109154fa5afbc6008d6a81984 2023-03-08 18:53:54,361 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/466b6af109154fa5afbc6008d6a81984, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 18:53:54,363 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8cccb8903f14f2a6b0b76a282ea1eb32 in 467ms, sequenceid=41, compaction requested=true 2023-03-08 18:53:54,363 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:54,363 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:54,363 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:53:54,363 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:53:54,364 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,364 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 18:53:54,364 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 18:53:54,365 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,365 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:54,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:54,366 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,366 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:53:54,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:54,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:54,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:54,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 18:53:54,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32453ff1[Count = 0] remaining members to acquire global barrier 2023-03-08 18:53:54,367 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 18:53:54,367 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,369 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,369 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:53:54,369 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 18:53:54,370 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 18:53:54,370 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:53:54,376 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 18:53:54,376 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:54,376 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:54,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:54,376 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 18:53:54,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:54,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:54,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:54,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:53:54,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 18:53:54,383 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 18:53:54,383 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:53:54,383 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:53:54,383 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,383 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:53:54,385 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,385 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,385 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:54,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:53:54,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:53:54,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:54,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:54,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:53:54,386 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:53:54,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,387 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:53:54,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:53:54,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 18:53:54,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:53:54,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:53:54,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:53:54,390 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 18:53:54,390 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 18:53:54,391 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:53:54,390 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:53:54,391 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,491 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:53:54,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:53:54,497 DEBUG [Listener at localhost/36281] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 18:53:54,500 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:43222, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 18:53:54,501 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:54,513 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 18:53:54,514 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 18:53:54,515 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:53:54,515 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302234515 2023-03-08 18:53:54,517 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:53:54,522 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-08 18:53:54,522 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:54,522 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x50864191 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:53:54,527 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@239158a2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:53:54,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,007 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010338297725b9fe4a58a9fe63a2fe194f45 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010338297725b9fe4a58a9fe63a2fe194f45 2023-03-08 18:53:55,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:55,027 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:53:55,029 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:53746, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:53:55,040 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:53:55,042 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:43234, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:53:55,077 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1d69c79209914727936df686cae1121d first=Optional[r3] last=Optional[r6] 2023-03-08 18:53:55,092 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1d69c79209914727936df686cae1121d}] 2023-03-08 18:53:55,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1d69c79209914727936df686cae1121d for inclusion in 8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:55,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 18:53:55,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:53:55,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:55,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1d69c79209914727936df686cae1121d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gt4ngc6gdegiqohuqlpvfn26k9p5pdciue94jk1gtnmcmm1m0qaqshkjpnl0tqf7/family1/1d69c79209914727936df686cae1121d 2023-03-08 18:53:55,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gt4ngc6gdegiqohuqlpvfn26k9p5pdciue94jk1gtnmcmm1m0qaqshkjpnl0tqf7/family1/1d69c79209914727936df686cae1121d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/959aae93a86a43f084cc2efcf718f6c5_SeqId_44_ 2023-03-08 18:53:55,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gt4ngc6gdegiqohuqlpvfn26k9p5pdciue94jk1gtnmcmm1m0qaqshkjpnl0tqf7/family1/1d69c79209914727936df686cae1121d into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/959aae93a86a43f084cc2efcf718f6c5_SeqId_44_ - updating store file list. 2023-03-08 18:53:55,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/959aae93a86a43f084cc2efcf718f6c5_SeqId_44_ into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:55,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gt4ngc6gdegiqohuqlpvfn26k9p5pdciue94jk1gtnmcmm1m0qaqshkjpnl0tqf7/family1/1d69c79209914727936df686cae1121d into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/959aae93a86a43f084cc2efcf718f6c5_SeqId_44_) 2023-03-08 18:53:55,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gt4ngc6gdegiqohuqlpvfn26k9p5pdciue94jk1gtnmcmm1m0qaqshkjpnl0tqf7/family1/1d69c79209914727936df686cae1121d 2023-03-08 18:53:55,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 18:53:55,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 18:53:55,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): 8cccb8903f14f2a6b0b76a282ea1eb32/family1 is initiating minor compaction 2023-03-08 18:53:55,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@70bffa5e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:53:55,144 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8cccb8903f14f2a6b0b76a282ea1eb32/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:55,145 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/8317d72e8fec41c98ac2b6478d34472a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/73500d03b3dc4f75be8664f1bff19ffd, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/89366c7c71584f128101bf3ee6476bb4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/ea1103c310e94cb4b1e51b810f6fd1b9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/88f7dfc78d3c430c9d661d5cd577d78b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e639902ff47a4b549f8ebfde6fb693c8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/70ba943aa9fd41b39c910889995d919e, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/c97a37cb7b7a46b8be89f0817f015fe7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/466b6af109154fa5afbc6008d6a81984, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/959aae93a86a43f084cc2efcf718f6c5_SeqId_44_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp, totalSize=50.2 K 2023-03-08 18:53:55,145 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 8317d72e8fec41c98ac2b6478d34472a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 18:53:55,145 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 73500d03b3dc4f75be8664f1bff19ffd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 18:53:55,146 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 89366c7c71584f128101bf3ee6476bb4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 18:53:55,146 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting ea1103c310e94cb4b1e51b810f6fd1b9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 18:53:55,146 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 88f7dfc78d3c430c9d661d5cd577d78b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 18:53:55,146 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting e639902ff47a4b549f8ebfde6fb693c8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 18:53:55,146 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 70ba943aa9fd41b39c910889995d919e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 18:53:55,146 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting c97a37cb7b7a46b8be89f0817f015fe7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 18:53:55,146 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 466b6af109154fa5afbc6008d6a81984, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 18:53:55,146 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 959aae93a86a43f084cc2efcf718f6c5_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 18:53:55,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:55,163 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:53:55,169 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228cc7a7af3fb404ce9a11d67c4b7a8587c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228cc7a7af3fb404ce9a11d67c4b7a8587c 2023-03-08 18:53:55,171 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231db9179a0c667499783b85d46dcee3fd3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231db9179a0c667499783b85d46dcee3fd3 2023-03-08 18:53:55,173 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b36e346a283f4b1a9474ed10065aceb1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b36e346a283f4b1a9474ed10065aceb1 2023-03-08 18:53:55,175 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010323a8c14054634e849b8299aa59cad29f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010323a8c14054634e849b8299aa59cad29f 2023-03-08 18:53:55,175 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-08 18:53:55,175 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 18:53:55,176 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,176 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,176 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,176 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,177 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,177 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,177 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,177 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:55,203 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8cccb8903f14f2a6b0b76a282ea1eb32#family1#compaction#10 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:53:56,031 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/0423cab510ab49a4b36302fa4a375da7 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0423cab510ab49a4b36302fa4a375da7 2023-03-08 18:53:56,051 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 8cccb8903f14f2a6b0b76a282ea1eb32/family1 of 8cccb8903f14f2a6b0b76a282ea1eb32 into 0423cab510ab49a4b36302fa4a375da7(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:53:56,051 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:56,053 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1678301635141; duration=0sec 2023-03-08 18:53:56,054 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:53:56,071 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f14a881f927945019ea61b03a8295960 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205f14a881f927945019ea61b03a8295960 2023-03-08 18:53:56,074 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:56,095 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/8317d72e8fec41c98ac2b6478d34472a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/73500d03b3dc4f75be8664f1bff19ffd, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/89366c7c71584f128101bf3ee6476bb4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/ea1103c310e94cb4b1e51b810f6fd1b9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/88f7dfc78d3c430c9d661d5cd577d78b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e639902ff47a4b549f8ebfde6fb693c8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/70ba943aa9fd41b39c910889995d919e, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/c97a37cb7b7a46b8be89f0817f015fe7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/466b6af109154fa5afbc6008d6a81984, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/959aae93a86a43f084cc2efcf718f6c5_SeqId_44_] to archive 2023-03-08 18:53:56,098 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:53:56,101 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/8317d72e8fec41c98ac2b6478d34472a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/8317d72e8fec41c98ac2b6478d34472a 2023-03-08 18:53:56,104 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/73500d03b3dc4f75be8664f1bff19ffd to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/73500d03b3dc4f75be8664f1bff19ffd 2023-03-08 18:53:56,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/89366c7c71584f128101bf3ee6476bb4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/89366c7c71584f128101bf3ee6476bb4 2023-03-08 18:53:56,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/ea1103c310e94cb4b1e51b810f6fd1b9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/ea1103c310e94cb4b1e51b810f6fd1b9 2023-03-08 18:53:56,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/88f7dfc78d3c430c9d661d5cd577d78b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/88f7dfc78d3c430c9d661d5cd577d78b 2023-03-08 18:53:56,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e639902ff47a4b549f8ebfde6fb693c8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e639902ff47a4b549f8ebfde6fb693c8 2023-03-08 18:53:56,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/70ba943aa9fd41b39c910889995d919e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/70ba943aa9fd41b39c910889995d919e 2023-03-08 18:53:56,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/c97a37cb7b7a46b8be89f0817f015fe7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/c97a37cb7b7a46b8be89f0817f015fe7 2023-03-08 18:53:56,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/466b6af109154fa5afbc6008d6a81984 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/466b6af109154fa5afbc6008d6a81984 2023-03-08 18:53:56,125 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9bb23b748dff4b2cb8797f0afcbd425a first=Optional[r00] last=Optional[r2] 2023-03-08 18:53:56,126 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9bb23b748dff4b2cb8797f0afcbd425a}] 2023-03-08 18:53:56,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/959aae93a86a43f084cc2efcf718f6c5_SeqId_44_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/959aae93a86a43f084cc2efcf718f6c5_SeqId_44_ 2023-03-08 18:53:56,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9bb23b748dff4b2cb8797f0afcbd425a for inclusion in 8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:56,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 18:53:56,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:53:56,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:56,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9bb23b748dff4b2cb8797f0afcbd425a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjs6qtiiva761uu2p663sdsom1jpteh860auts9git36npt82t9r7us11ls36a0t/family1/9bb23b748dff4b2cb8797f0afcbd425a 2023-03-08 18:53:56,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjs6qtiiva761uu2p663sdsom1jpteh860auts9git36npt82t9r7us11ls36a0t/family1/9bb23b748dff4b2cb8797f0afcbd425a as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/9cc11297889248249a1c1d3a62570e3b_SeqId_47_ 2023-03-08 18:53:56,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjs6qtiiva761uu2p663sdsom1jpteh860auts9git36npt82t9r7us11ls36a0t/family1/9bb23b748dff4b2cb8797f0afcbd425a into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/9cc11297889248249a1c1d3a62570e3b_SeqId_47_ - updating store file list. 2023-03-08 18:53:56,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/9cc11297889248249a1c1d3a62570e3b_SeqId_47_ into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:56,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjs6qtiiva761uu2p663sdsom1jpteh860auts9git36npt82t9r7us11ls36a0t/family1/9bb23b748dff4b2cb8797f0afcbd425a into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/9cc11297889248249a1c1d3a62570e3b_SeqId_47_) 2023-03-08 18:53:56,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjs6qtiiva761uu2p663sdsom1jpteh860auts9git36npt82t9r7us11ls36a0t/family1/9bb23b748dff4b2cb8797f0afcbd425a 2023-03-08 18:53:56,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 18:53:56,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 18:53:56,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): 8cccb8903f14f2a6b0b76a282ea1eb32/family1 is initiating minor compaction (all files) 2023-03-08 18:53:56,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6b27ad7b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:53:56,155 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8cccb8903f14f2a6b0b76a282ea1eb32/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:56,156 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e66a380ac8304766b653c08cd34daa2d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0423cab510ab49a4b36302fa4a375da7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/9cc11297889248249a1c1d3a62570e3b_SeqId_47_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp, totalSize=358.6 K 2023-03-08 18:53:56,157 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting e66a380ac8304766b653c08cd34daa2d, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845940 2023-03-08 18:53:56,158 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 0423cab510ab49a4b36302fa4a375da7, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245940 2023-03-08 18:53:56,158 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 9cc11297889248249a1c1d3a62570e3b_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845940 2023-03-08 18:53:56,159 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:56,160 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:53:56,162 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130cde836015ea84cba86f44f34a8346775 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130cde836015ea84cba86f44f34a8346775 2023-03-08 18:53:56,164 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512014e363fc553d74137a5ba10ecd59a938d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512014e363fc553d74137a5ba10ecd59a938d 2023-03-08 18:53:56,166 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205ab2b8696af054d3a8f3a6c647f4e452a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205ab2b8696af054d3a8f3a6c647f4e452a 2023-03-08 18:53:56,166 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-08 18:53:56,166 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 18:53:56,167 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:56,167 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:56,168 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:53:56,170 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 18:53:56,172 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.6640 sec 2023-03-08 18:53:56,203 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8cccb8903f14f2a6b0b76a282ea1eb32#family1#compaction#11 average throughput is 6.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:53:56,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$12(2454): Client=jenkins//172.31.2.20 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 18:53:56,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:56,782 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-08 18:53:56,784 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-08 18:53:56,784 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-08 18:53:56,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-08 18:53:56,801 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8cccb8903f14f2a6b0b76a282ea1eb32, REOPEN/MOVE}] 2023-03-08 18:53:56,802 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8cccb8903f14f2a6b0b76a282ea1eb32, REOPEN/MOVE 2023-03-08 18:53:56,803 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=8cccb8903f14f2a6b0b76a282ea1eb32, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:56,803 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678301636803"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301636803"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301636803"}]},"ts":"1678301636803"} 2023-03-08 18:53:56,805 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 8cccb8903f14f2a6b0b76a282ea1eb32, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:53:56,964 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:56,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 8cccb8903f14f2a6b0b76a282ea1eb32, disabling compactions & flushes 2023-03-08 18:53:56,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:57,453 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/.tmp/family1/777658cb216d41eea7dde92fc36e5023 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/777658cb216d41eea7dde92fc36e5023 2023-03-08 18:53:57,460 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 8cccb8903f14f2a6b0b76a282ea1eb32/family1 of 8cccb8903f14f2a6b0b76a282ea1eb32 into 777658cb216d41eea7dde92fc36e5023(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:53:57,460 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:57,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:57,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:57,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. after waiting 0 ms 2023-03-08 18:53:57,461 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1678301636155; duration=1sec 2023-03-08 18:53:57,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:57,461 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:53:57,463 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e66a380ac8304766b653c08cd34daa2d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0423cab510ab49a4b36302fa4a375da7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/9cc11297889248249a1c1d3a62570e3b_SeqId_47_] to archive 2023-03-08 18:53:57,464 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:53:57,467 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e66a380ac8304766b653c08cd34daa2d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/e66a380ac8304766b653c08cd34daa2d 2023-03-08 18:53:57,468 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0423cab510ab49a4b36302fa4a375da7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0423cab510ab49a4b36302fa4a375da7 2023-03-08 18:53:57,470 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/9cc11297889248249a1c1d3a62570e3b_SeqId_47_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/9cc11297889248249a1c1d3a62570e3b_SeqId_47_ 2023-03-08 18:53:57,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 18:53:57,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:57,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:57,478 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 8cccb8903f14f2a6b0b76a282ea1eb32 to self. 2023-03-08 18:53:57,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:57,481 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=8cccb8903f14f2a6b0b76a282ea1eb32, regionState=CLOSED 2023-03-08 18:53:57,481 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678301637481"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301637481"}]},"ts":"1678301637481"} 2023-03-08 18:53:57,485 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-08 18:53:57,485 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 8cccb8903f14f2a6b0b76a282ea1eb32, server=jenkins-hbase6.apache.org,46397,1678301621627 in 678 msec 2023-03-08 18:53:57,486 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8cccb8903f14f2a6b0b76a282ea1eb32, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=true 2023-03-08 18:53:57,636 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=8cccb8903f14f2a6b0b76a282ea1eb32, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:57,637 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678301637636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301637636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301637636"}]},"ts":"1678301637636"} 2023-03-08 18:53:57,639 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 8cccb8903f14f2a6b0b76a282ea1eb32, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:53:57,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:57,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8cccb8903f14f2a6b0b76a282ea1eb32, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:53:57,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:57,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:57,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:57,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:57,797 INFO [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:57,799 DEBUG [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:57,799 DEBUG [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:57,799 INFO [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8cccb8903f14f2a6b0b76a282ea1eb32 columnFamilyName family1 2023-03-08 18:53:57,806 DEBUG [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/777658cb216d41eea7dde92fc36e5023 2023-03-08 18:53:57,807 INFO [StoreOpener-8cccb8903f14f2a6b0b76a282ea1eb32-1] regionserver.HStore(310): Store=8cccb8903f14f2a6b0b76a282ea1eb32/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:57,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:57,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:57,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:53:57,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 8cccb8903f14f2a6b0b76a282ea1eb32; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67439843, jitterRate=0.004931971430778503}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:53:57,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:57,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32., pid=17, masterSystemTime=1678301637791 2023-03-08 18:53:57,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:57,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:53:57,816 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=8cccb8903f14f2a6b0b76a282ea1eb32, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:57,817 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678301637816"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301637816"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301637816"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301637816"}]},"ts":"1678301637816"} 2023-03-08 18:53:57,821 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-08 18:53:57,821 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 8cccb8903f14f2a6b0b76a282ea1eb32, server=jenkins-hbase6.apache.org,46397,1678301621627 in 180 msec 2023-03-08 18:53:57,823 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-08 18:53:57,823 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8cccb8903f14f2a6b0b76a282ea1eb32, REOPEN/MOVE in 1.0200 sec 2023-03-08 18:53:57,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-08 18:53:57,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0370 sec 2023-03-08 18:53:57,829 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4760 sec 2023-03-08 18:53:57,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-08 18:53:57,841 INFO [Listener at localhost/36281] compactions.TestMobCompactor(278): alter status finished 2023-03-08 18:53:57,842 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 18:53:57,844 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 18:53:57,845 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 18:53:57,846 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:53:57,846 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302237846 2023-03-08 18:53:57,847 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:53:57,847 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-08 18:53:57,847 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:57,848 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x0f95eb0f to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:53:57,912 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56473439, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:53:57,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:57,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:57,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:57,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:57,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:58,347 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289e6c5e226a6545ff94acb40da918a4a7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289e6c5e226a6545ff94acb40da918a4a7 2023-03-08 18:53:58,349 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:58,366 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:53:58,368 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:53762, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:53:58,376 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:53:58,378 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:43242, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:53:58,389 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/105e70946fb84ab987c2351f6a505150 first=Optional[r05] last=Optional[r8] 2023-03-08 18:53:58,390 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=51 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/105e70946fb84ab987c2351f6a505150}] 2023-03-08 18:53:58,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/105e70946fb84ab987c2351f6a505150 for inclusion in 8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:58,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 18:53:58,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:53:58,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:53:58,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/105e70946fb84ab987c2351f6a505150 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__pnkqtc2o5hth6prkphqe42vg5r2ptfuld74o178s9bfglst1k5jo41s39a0due3j/family1/105e70946fb84ab987c2351f6a505150 2023-03-08 18:53:58,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__pnkqtc2o5hth6prkphqe42vg5r2ptfuld74o178s9bfglst1k5jo41s39a0due3j/family1/105e70946fb84ab987c2351f6a505150 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0e2a6aa9cb5442e7acf8ebbdb4a03481_SeqId_53_ 2023-03-08 18:53:58,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__pnkqtc2o5hth6prkphqe42vg5r2ptfuld74o178s9bfglst1k5jo41s39a0due3j/family1/105e70946fb84ab987c2351f6a505150 into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0e2a6aa9cb5442e7acf8ebbdb4a03481_SeqId_53_ - updating store file list. 2023-03-08 18:53:58,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0e2a6aa9cb5442e7acf8ebbdb4a03481_SeqId_53_ into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 2023-03-08 18:53:58,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__pnkqtc2o5hth6prkphqe42vg5r2ptfuld74o178s9bfglst1k5jo41s39a0due3j/family1/105e70946fb84ab987c2351f6a505150 into 8cccb8903f14f2a6b0b76a282ea1eb32/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/family1/0e2a6aa9cb5442e7acf8ebbdb4a03481_SeqId_53_) 2023-03-08 18:53:58,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__pnkqtc2o5hth6prkphqe42vg5r2ptfuld74o178s9bfglst1k5jo41s39a0due3j/family1/105e70946fb84ab987c2351f6a505150 2023-03-08 18:53:58,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:58,414 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:53:58,415 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a99f1094860e4690b76efc5bdc00d4d6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a99f1094860e4690b76efc5bdc00d4d6 2023-03-08 18:53:58,417 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128e567a85f279d40bb9cf636892762a31d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128e567a85f279d40bb9cf636892762a31d 2023-03-08 18:53:58,417 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-08 18:53:58,417 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 18:53:58,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:58,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:58,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 18:53:58,418 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 18:53:58,418 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:53:58,419 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 18:53:58,421 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 576 msec 2023-03-08 18:53:58,509 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=248 (was 211) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x50864191-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x0f95eb0f sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1420081506.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_703111363_15 at /127.0.0.1:57708 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57656 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57676 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57510 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x39f4371b-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x0f95eb0f-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57732 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37d7fc7a-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x0f95eb0f-SendThread(127.0.0.1:54098) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57556 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57538 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x50864191-SendThread(127.0.0.1:54098) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57546 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57488 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37d7fc7a-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57562 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57642 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57606 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x39f4371b-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:42764 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57698 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37d7fc7a-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x50864191 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1420081506.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57558 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase6:46397-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:57740 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=593 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=242 (was 311), ProcessCount=187 (was 184) - ProcessCount LEAK? -, AvailableMemoryMB=7052 (was 7393) 2023-03-08 18:53:58,523 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=593, MaxFileDescriptor=60000, SystemLoadAverage=242, ProcessCount=187, AvailableMemoryMB=7052 2023-03-08 18:53:58,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 18:53:58,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-08 18:53:58,528 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:53:58,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-08 18:53:58,529 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:53:58,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 18:53:58,531 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d 2023-03-08 18:53:58,532 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d empty. 2023-03-08 18:53:58,532 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d 2023-03-08 18:53:58,532 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-08 18:53:58,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 18:53:58,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 18:53:58,942 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 18:53:58,943 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8336865ae81041d039911221d7ad790d, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:53:59,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 18:53:59,350 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:59,350 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 8336865ae81041d039911221d7ad790d, disabling compactions & flushes 2023-03-08 18:53:59,350 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:53:59,350 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:53:59,350 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. after waiting 0 ms 2023-03-08 18:53:59,350 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:53:59,350 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:53:59,351 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:53:59,353 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:53:59,354 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678301639354"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301639354"}]},"ts":"1678301639354"} 2023-03-08 18:53:59,356 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:53:59,357 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:53:59,357 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301639357"}]},"ts":"1678301639357"} 2023-03-08 18:53:59,359 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-08 18:53:59,450 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=8336865ae81041d039911221d7ad790d, ASSIGN}] 2023-03-08 18:53:59,452 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=8336865ae81041d039911221d7ad790d, ASSIGN 2023-03-08 18:53:59,453 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=8336865ae81041d039911221d7ad790d, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:53:59,604 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=8336865ae81041d039911221d7ad790d, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:59,604 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678301639604"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301639604"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301639604"}]},"ts":"1678301639604"} 2023-03-08 18:53:59,606 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 8336865ae81041d039911221d7ad790d, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:53:59,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 18:53:59,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:53:59,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8336865ae81041d039911221d7ad790d, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:53:59,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 8336865ae81041d039911221d7ad790d 2023-03-08 18:53:59,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:53:59,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 8336865ae81041d039911221d7ad790d 2023-03-08 18:53:59,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 8336865ae81041d039911221d7ad790d 2023-03-08 18:53:59,764 INFO [StoreOpener-8336865ae81041d039911221d7ad790d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8336865ae81041d039911221d7ad790d 2023-03-08 18:53:59,766 DEBUG [StoreOpener-8336865ae81041d039911221d7ad790d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1 2023-03-08 18:53:59,766 DEBUG [StoreOpener-8336865ae81041d039911221d7ad790d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1 2023-03-08 18:53:59,766 INFO [StoreOpener-8336865ae81041d039911221d7ad790d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8336865ae81041d039911221d7ad790d columnFamilyName family1 2023-03-08 18:53:59,768 INFO [StoreOpener-8336865ae81041d039911221d7ad790d-1] regionserver.HStore(310): Store=8336865ae81041d039911221d7ad790d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:53:59,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d 2023-03-08 18:53:59,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d 2023-03-08 18:53:59,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 8336865ae81041d039911221d7ad790d 2023-03-08 18:53:59,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:53:59,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 8336865ae81041d039911221d7ad790d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59101501, jitterRate=-0.11931900680065155}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:53:59,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:53:59,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d., pid=21, masterSystemTime=1678301639758 2023-03-08 18:53:59,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:53:59,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:53:59,780 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=8336865ae81041d039911221d7ad790d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:53:59,780 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678301639779"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301639779"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301639779"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301639779"}]},"ts":"1678301639779"} 2023-03-08 18:53:59,783 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-08 18:53:59,784 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 8336865ae81041d039911221d7ad790d, server=jenkins-hbase6.apache.org,46397,1678301621627 in 175 msec 2023-03-08 18:53:59,785 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-08 18:53:59,786 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=8336865ae81041d039911221d7ad790d, ASSIGN in 333 msec 2023-03-08 18:53:59,786 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:53:59,786 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301639786"}]},"ts":"1678301639786"} 2023-03-08 18:53:59,788 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-08 18:53:59,859 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:53:59,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.3330 sec 2023-03-08 18:54:00,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 18:54:00,637 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-08 18:54:00,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:00,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:00,680 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:00,680 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:00,681 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:00,681 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:00,681 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:00,681 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:00,734 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:00,734 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:00,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:00,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:00,735 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:00,735 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:00,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:00,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:00,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:00,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:00,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:00,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:00,736 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:00,736 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:00,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:00,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:00,737 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:00,737 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:00,737 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:00,738 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 18:54:01,162 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307a3dbca807a94cf0a98e988e0f643f19 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511307a3dbca807a94cf0a98e988e0f643f19 2023-03-08 18:54:01,166 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/7246705f57b8489ab3c8fa6fd877fa02 2023-03-08 18:54:01,177 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/7246705f57b8489ab3c8fa6fd877fa02 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7246705f57b8489ab3c8fa6fd877fa02 2023-03-08 18:54:01,185 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7246705f57b8489ab3c8fa6fd877fa02, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-08 18:54:01,186 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 449ms, sequenceid=5, compaction requested=false 2023-03-08 18:54:01,187 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:01,187 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:01,188 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:01,188 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:01,188 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:01,188 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,188 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:01,188 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:01,250 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,250 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:01,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:01,250 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,251 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:01,252 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fbafd04[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:01,252 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,289 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,289 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:01,289 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:01,289 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:01,289 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,290 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:01,350 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:01,350 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,350 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:01,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:01,350 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:01,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:01,353 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:01,353 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:01,353 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:01,353 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,353 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:01,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:01,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:01,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:01,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:01,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:01,413 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:01,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:01,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,414 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:01,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,481 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:01,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:01,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:54:01,481 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:01,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:01,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:01,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:01,481 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:01,481 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,482 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,482 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,482 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:01,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:01,482 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,482 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:01,483 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,483 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,483 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,583 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:01,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:01,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:01,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:01,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,591 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:01,591 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:01,592 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:01,592 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:01,592 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,592 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,642 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,642 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:01,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:01,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:01,643 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:01,643 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:01,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:01,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,644 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,644 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:01,644 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:01,644 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:01,644 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:01,645 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:01,645 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:01,645 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:01,645 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:01,645 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:01,645 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 18:54:02,060 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285a366342f3e146549d1b0ae77a4260a4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511285a366342f3e146549d1b0ae77a4260a4 2023-03-08 18:54:02,467 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/6d7a92498d4847babd456d62ad2092eb 2023-03-08 18:54:02,473 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/6d7a92498d4847babd456d62ad2092eb as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/6d7a92498d4847babd456d62ad2092eb 2023-03-08 18:54:02,479 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/6d7a92498d4847babd456d62ad2092eb, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 18:54:02,480 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 835ms, sequenceid=9, compaction requested=false 2023-03-08 18:54:02,481 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:02,481 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:02,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:02,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:02,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:02,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:02,534 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,534 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:02,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:02,535 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:02,535 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:02,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:02,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:02,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:02,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@138aacc4[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:02,537 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:02,537 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,611 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,612 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:02,612 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:02,612 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:02,612 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,612 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:02,665 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:02,665 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,665 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:02,666 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:02,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:02,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:02,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:02,669 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:02,669 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:02,669 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:02,669 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:02,669 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,669 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:02,712 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,712 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:02,712 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:02,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:02,713 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:02,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,714 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:02,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,800 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,800 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:02,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:02,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:02,800 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,800 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:02,800 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,800 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:02,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:02,800 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:02,801 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:02,801 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:02,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:02,801 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,801 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,801 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:02,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:02,901 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:02,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:02,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:02,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:02,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,909 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:02,909 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:02,910 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:02,910 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:02,910 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:02,910 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,010 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:54:03,011 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:03,011 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:03,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:03,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:03,012 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:03,012 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:03,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:03,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,013 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,013 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:03,013 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,013 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:03,013 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:03,014 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:03,014 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:03,014 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:03,014 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:03,014 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:03,014 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:03,430 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512018e0be4093bd64f7db42fda32fdecab95 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512018e0be4093bd64f7db42fda32fdecab95 2023-03-08 18:54:03,836 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/081f8d562e674526a8ab8f92dab93080 2023-03-08 18:54:03,843 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/081f8d562e674526a8ab8f92dab93080 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/081f8d562e674526a8ab8f92dab93080 2023-03-08 18:54:03,848 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/081f8d562e674526a8ab8f92dab93080, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 18:54:03,850 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 836ms, sequenceid=13, compaction requested=true 2023-03-08 18:54:03,850 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:03,850 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:03,850 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:03,850 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:03,850 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:03,850 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:03,850 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:03,904 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:03,904 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:03,904 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:03,904 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:03,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:03,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:03,905 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:03,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5803aef2[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:03,906 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:03,906 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,990 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:03,990 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:03,990 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:03,990 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:03,990 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:03,990 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:04,042 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:04,042 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,042 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,043 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:04,046 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:04,046 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:04,046 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:04,046 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,046 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:04,104 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,104 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:04,104 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,105 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:04,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,106 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,165 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,165 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:04,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:04,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:04,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:54:04,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:04,165 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:04,165 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:04,165 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,166 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:04,166 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:04,166 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,166 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:04,167 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,167 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,167 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:04,267 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:04,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:04,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:04,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:04,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,277 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:04,277 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:04,278 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:04,278 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:04,279 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,279 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,343 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,343 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:04,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:04,344 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:04,344 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,345 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:04,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:04,345 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:04,345 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:04,345 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:04,346 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:04,346 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:04,346 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:04,346 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:04,346 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:04,761 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120532f36b7a77c243dc9fc00e8d9577286c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120532f36b7a77c243dc9fc00e8d9577286c 2023-03-08 18:54:05,167 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/0b5558a0a5b94b14a281684e168ca3d0 2023-03-08 18:54:05,174 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/0b5558a0a5b94b14a281684e168ca3d0 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/0b5558a0a5b94b14a281684e168ca3d0 2023-03-08 18:54:05,179 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/0b5558a0a5b94b14a281684e168ca3d0, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 18:54:05,180 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 834ms, sequenceid=17, compaction requested=true 2023-03-08 18:54:05,180 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:05,180 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:05,180 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:05,180 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:05,180 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,180 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:05,180 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:05,182 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,182 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:05,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:05,182 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,182 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:05,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:05,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:05,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4322b8b4[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:05,184 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:05,184 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,185 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,185 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:05,185 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,185 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:05,185 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:05,185 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:05,187 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,187 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:05,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:05,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:05,187 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:05,187 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:05,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:05,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:05,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:05,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:05,189 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:05,189 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:05,190 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:05,190 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:05,190 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,190 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:05,192 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,192 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:05,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:05,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,192 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:05,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:05,193 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:05,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:05,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:05,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,194 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:05,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,197 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,197 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,197 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,197 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,197 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:05,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:54:05,197 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:05,198 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,197 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:05,198 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:05,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:05,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:05,198 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:05,198 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:05,198 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:05,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:05,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:05,298 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:05,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:05,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:05,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:05,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,307 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:05,307 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:05,308 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:05,308 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:05,308 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,308 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,311 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,311 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:05,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:05,311 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,311 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,312 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,312 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:05,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,312 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:05,312 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:05,313 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:05,313 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:05,313 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:05,313 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:05,313 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:05,313 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:05,329 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228bc5110cf15444606a27faf9d7d52d7fc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228bc5110cf15444606a27faf9d7d52d7fc 2023-03-08 18:54:05,735 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/7af813807fbb4b038a5fcd83ab0df741 2023-03-08 18:54:05,742 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/7af813807fbb4b038a5fcd83ab0df741 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7af813807fbb4b038a5fcd83ab0df741 2023-03-08 18:54:05,747 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7af813807fbb4b038a5fcd83ab0df741, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 18:54:05,748 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 435ms, sequenceid=21, compaction requested=true 2023-03-08 18:54:05,749 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:05,749 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:05,749 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:05,749 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:05,749 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,749 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:05,749 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:05,751 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,751 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:05,751 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,751 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:05,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:05,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bc83f09[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:05,752 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:05,753 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,759 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,759 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:05,759 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:05,759 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:05,760 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,760 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:05,761 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,761 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:05,761 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:05,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:05,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:05,761 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:05,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:05,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:05,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:05,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:05,764 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:05,764 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:05,764 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:05,764 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:05,764 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,764 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:05,770 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,770 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:05,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,770 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:05,770 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:05,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:05,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:05,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,772 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:05,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,775 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,775 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:05,775 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,775 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:05,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:05,775 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:05,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:05,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:05,775 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,776 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,776 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:05,776 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:05,776 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:05,776 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,776 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:05,876 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:05,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:05,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:05,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:05,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,884 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:05,884 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:05,885 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:05,885 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:05,885 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,885 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,887 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:05,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:05,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:05,887 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:05,887 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:05,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:05,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:05,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:05,888 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:05,888 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:05,889 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:05,889 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:05,889 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:05,889 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:05,889 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:05,889 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:06,308 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317fe5695344894640a98b35d093b5d9d4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317fe5695344894640a98b35d093b5d9d4 2023-03-08 18:54:06,326 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/3ded35621e8a4cb38a47c6fb47fc05f7 2023-03-08 18:54:06,333 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/3ded35621e8a4cb38a47c6fb47fc05f7 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/3ded35621e8a4cb38a47c6fb47fc05f7 2023-03-08 18:54:06,340 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/3ded35621e8a4cb38a47c6fb47fc05f7, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 18:54:06,341 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 452ms, sequenceid=25, compaction requested=true 2023-03-08 18:54:06,341 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:06,341 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:06,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:06,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:06,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:06,341 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:06,343 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,343 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:06,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:06,343 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,343 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:06,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:06,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:06,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:06,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:06,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3de24038[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:06,345 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:06,345 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,346 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,347 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:06,347 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:06,347 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:06,347 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,347 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:06,349 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:06,349 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,349 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:06,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:06,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:06,349 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:06,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:06,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:06,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:06,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:06,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:06,353 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:06,353 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:06,353 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:06,353 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,353 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:06,355 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,355 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:06,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:06,355 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:06,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:06,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:06,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:06,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,357 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:06,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,358 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:06,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,361 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,361 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:06,361 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:06,361 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:06,361 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,361 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:06,362 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:06,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:06,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:06,362 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,362 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,362 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,362 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:06,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:06,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:06,363 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:06,463 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:06,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:06,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:06,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:06,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,471 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:06,471 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:06,472 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:06,472 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:06,472 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,472 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,474 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:06,474 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:06,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:06,474 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:06,474 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:06,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:06,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,475 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:06,475 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:06,475 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:06,475 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:06,476 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:06,476 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:06,476 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:06,476 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:06,476 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:06,476 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:06,896 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101854f700cfec0453cb4a841535217e133 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101854f700cfec0453cb4a841535217e133 2023-03-08 18:54:07,303 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/c53861a2a0ad46aabab7365114610ea4 2023-03-08 18:54:07,310 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/c53861a2a0ad46aabab7365114610ea4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/c53861a2a0ad46aabab7365114610ea4 2023-03-08 18:54:07,316 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/c53861a2a0ad46aabab7365114610ea4, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 18:54:07,317 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 841ms, sequenceid=29, compaction requested=true 2023-03-08 18:54:07,317 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:07,317 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:07,317 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:07,317 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:07,317 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,317 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:07,317 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:07,320 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,320 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:07,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:07,320 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,320 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:07,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:07,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:07,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:07,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:07,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78530d48[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:07,322 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:07,322 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,323 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,323 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:07,324 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:07,324 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,324 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:07,324 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:07,325 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:07,325 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:07,325 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,325 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:07,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:07,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:07,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:07,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:07,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:07,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:07,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:07,328 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:07,328 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:07,328 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:07,328 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,329 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:07,330 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,330 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,330 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:07,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:07,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:07,331 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:07,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:07,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:07,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:07,332 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:07,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,340 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:07,340 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,340 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:07,340 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:07,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:07,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:07,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:07,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:07,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:54:07,340 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,341 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:07,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:07,341 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,341 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,341 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,341 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:07,341 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:07,341 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,441 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:07,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:07,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:07,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:07,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,452 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:07,452 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:07,453 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:07,453 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:07,453 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,453 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,455 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,455 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:07,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:07,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:07,455 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,455 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:07,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:07,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,456 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,456 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:07,456 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,457 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:07,457 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:07,457 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:07,457 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:07,458 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:07,458 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:07,458 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:07,458 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:07,481 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103fc12c611af694749a7ec47f005d63e90 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103fc12c611af694749a7ec47f005d63e90 2023-03-08 18:54:07,888 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/016af30a77df4404b4b350de662d2c92 2023-03-08 18:54:07,895 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/016af30a77df4404b4b350de662d2c92 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016af30a77df4404b4b350de662d2c92 2023-03-08 18:54:07,901 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016af30a77df4404b4b350de662d2c92, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 18:54:07,902 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 444ms, sequenceid=33, compaction requested=true 2023-03-08 18:54:07,902 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:07,902 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:07,902 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:07,902 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:07,903 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,903 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:07,903 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:07,905 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,905 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:07,905 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,905 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:07,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:07,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:07,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ee2edee[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:07,907 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:07,907 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,908 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,908 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:07,908 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:07,908 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:07,908 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,908 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:07,914 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:07,914 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,914 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:07,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:07,914 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:07,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:07,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:07,917 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:07,917 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:07,917 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:07,917 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,917 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:07,919 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,919 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:07,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:07,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,919 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:07,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:07,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:07,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:07,920 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:07,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,921 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:07,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,923 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,923 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:07,923 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,923 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,923 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:07,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:07,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:07,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:07,923 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:07,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:07,924 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:07,924 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,924 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,925 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:07,925 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:07,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:07,925 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:07,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:08,025 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:08,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:08,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:08,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:08,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,036 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:08,036 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:08,036 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:08,036 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:08,037 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,037 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,038 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,038 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:08,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:08,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:08,039 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,039 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:08,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:08,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,040 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:08,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,040 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:08,040 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:08,041 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:08,042 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:08,042 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:08,042 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:08,042 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:08,042 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:08,067 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090719927a20277f48f1852917a90f2035c7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015090719927a20277f48f1852917a90f2035c7 2023-03-08 18:54:08,475 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/db06713932a742188a662b227d287b38 2023-03-08 18:54:08,484 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/db06713932a742188a662b227d287b38 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/db06713932a742188a662b227d287b38 2023-03-08 18:54:08,492 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/db06713932a742188a662b227d287b38, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 18:54:08,493 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 451ms, sequenceid=37, compaction requested=true 2023-03-08 18:54:08,493 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:08,493 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:08,494 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:08,494 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:08,494 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,494 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:08,494 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:08,499 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,500 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:08,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:08,500 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,500 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:08,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:08,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:08,502 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:08,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1468bf4c[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:08,502 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,504 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,504 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:08,504 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:08,504 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,504 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:08,504 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:08,505 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:08,506 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:08,505 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:08,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:08,506 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:08,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:08,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:08,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:08,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:08,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:08,509 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:08,509 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:08,510 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:08,510 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,510 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:08,512 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,512 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:08,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,512 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:08,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,513 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,515 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,519 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,519 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:08,519 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:08,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:08,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:08,519 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:08,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:54:08,519 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:08,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:08,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:08,519 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,520 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,520 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,520 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,520 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,520 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:08,520 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:08,620 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:08,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:08,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:08,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:08,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,629 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:08,629 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:08,629 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:54:08,629 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:08,630 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,630 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,632 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,632 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:08,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:08,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:08,632 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:08,632 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:08,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,633 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:08,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,633 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,633 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:54:08,633 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:08,633 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:54:08,633 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:08,635 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:54:08,635 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:08,635 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:08,635 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:08,635 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. started... 2023-03-08 18:54:08,635 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8336865ae81041d039911221d7ad790d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:09,060 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112079b35b88fb9d4a829e89d8c50ecf6f0f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112079b35b88fb9d4a829e89d8c50ecf6f0f 2023-03-08 18:54:09,467 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/ae348ee094a84754b1fe7a5de5417383 2023-03-08 18:54:09,473 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/ae348ee094a84754b1fe7a5de5417383 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/ae348ee094a84754b1fe7a5de5417383 2023-03-08 18:54:09,478 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/ae348ee094a84754b1fe7a5de5417383, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 18:54:09,479 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8336865ae81041d039911221d7ad790d in 844ms, sequenceid=41, compaction requested=true 2023-03-08 18:54:09,479 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:09,479 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:09,480 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:09,480 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:09,480 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,480 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:54:09,480 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:54:09,481 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:09,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:09,482 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,482 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:09,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:09,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:09,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:09,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:54:09,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ba4de79[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:09,483 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:09,483 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,485 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,485 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:09,485 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:54:09,485 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:09,485 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,485 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:09,488 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:54:09,488 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:09,488 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,488 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:54:09,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:09,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:09,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:09,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:09,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:09,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:09,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:09,491 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:54:09,491 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:09,491 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:09,491 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,491 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:09,493 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,493 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,493 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:09,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:09,493 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,495 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:09,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:09,497 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:09,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:09,497 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:54:09,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 18:54:09,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:09,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:09,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:09,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:09,499 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,499 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,499 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:09,499 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:09,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:09,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:09,599 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:09,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:09,602 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-08 18:54:09,603 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:09,604 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:09,605 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:09,605 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302249605 2023-03-08 18:54:09,606 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:54:09,607 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-08 18:54:09,608 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:54:09,608 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x0a2ebee3 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:54:09,613 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6527a563, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:54:09,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:54:10,450 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010324a51648fac1464f97fc7e2662c0457a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010324a51648fac1464f97fc7e2662c0457a 2023-03-08 18:54:10,451 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:54:10,465 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:54:10,466 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:32890, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:54:10,473 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:54:10,474 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:33640, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:54:10,483 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/84870f0a33ac4ab4bdad5bc153ecec4b first=Optional[r5] last=Optional[r6] 2023-03-08 18:54:10,484 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/84870f0a33ac4ab4bdad5bc153ecec4b}] 2023-03-08 18:54:10,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/84870f0a33ac4ab4bdad5bc153ecec4b for inclusion in 8336865ae81041d039911221d7ad790d/family1 2023-03-08 18:54:10,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-08 18:54:10,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:10,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:10,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/84870f0a33ac4ab4bdad5bc153ecec4b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__c4bckdqqchaa6k7kgm61ns9rfj4vg2qe4mfkdfsnqb2705l2ea42k56vcofaruck/family1/84870f0a33ac4ab4bdad5bc153ecec4b 2023-03-08 18:54:10,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__c4bckdqqchaa6k7kgm61ns9rfj4vg2qe4mfkdfsnqb2705l2ea42k56vcofaruck/family1/84870f0a33ac4ab4bdad5bc153ecec4b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_ 2023-03-08 18:54:10,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__c4bckdqqchaa6k7kgm61ns9rfj4vg2qe4mfkdfsnqb2705l2ea42k56vcofaruck/family1/84870f0a33ac4ab4bdad5bc153ecec4b into 8336865ae81041d039911221d7ad790d/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_ - updating store file list. 2023-03-08 18:54:10,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_ into 8336865ae81041d039911221d7ad790d/family1 2023-03-08 18:54:10,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__c4bckdqqchaa6k7kgm61ns9rfj4vg2qe4mfkdfsnqb2705l2ea42k56vcofaruck/family1/84870f0a33ac4ab4bdad5bc153ecec4b into 8336865ae81041d039911221d7ad790d/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_) 2023-03-08 18:54:10,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__c4bckdqqchaa6k7kgm61ns9rfj4vg2qe4mfkdfsnqb2705l2ea42k56vcofaruck/family1/84870f0a33ac4ab4bdad5bc153ecec4b 2023-03-08 18:54:10,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 18:54:10,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 18:54:10,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): 8336865ae81041d039911221d7ad790d/family1 is initiating minor compaction 2023-03-08 18:54:10,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@12b575ae; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:10,505 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8336865ae81041d039911221d7ad790d/family1 in testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:10,505 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/6d7a92498d4847babd456d62ad2092eb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/081f8d562e674526a8ab8f92dab93080, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/0b5558a0a5b94b14a281684e168ca3d0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7af813807fbb4b038a5fcd83ab0df741, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/3ded35621e8a4cb38a47c6fb47fc05f7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/c53861a2a0ad46aabab7365114610ea4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016af30a77df4404b4b350de662d2c92, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/db06713932a742188a662b227d287b38, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/ae348ee094a84754b1fe7a5de5417383, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp, totalSize=49.7 K 2023-03-08 18:54:10,505 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 6d7a92498d4847babd456d62ad2092eb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 18:54:10,505 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 081f8d562e674526a8ab8f92dab93080, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 18:54:10,506 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 0b5558a0a5b94b14a281684e168ca3d0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 18:54:10,506 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 7af813807fbb4b038a5fcd83ab0df741, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 18:54:10,506 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 3ded35621e8a4cb38a47c6fb47fc05f7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 18:54:10,506 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting c53861a2a0ad46aabab7365114610ea4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 18:54:10,506 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 016af30a77df4404b4b350de662d2c92, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 18:54:10,506 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting db06713932a742188a662b227d287b38, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 18:54:10,506 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting ae348ee094a84754b1fe7a5de5417383, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 18:54:10,506 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 18:54:10,507 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:54:10,508 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:10,510 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101854f700cfec0453cb4a841535217e133 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101854f700cfec0453cb4a841535217e133 2023-03-08 18:54:10,511 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103fc12c611af694749a7ec47f005d63e90 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103fc12c611af694749a7ec47f005d63e90 2023-03-08 18:54:10,511 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-08 18:54:10,511 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-08 18:54:10,512 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:54:10,512 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:54:10,544 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8336865ae81041d039911221d7ad790d#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:10,963 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/cdeb3a8bdc45426b9d14abca7ed52c1a as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/cdeb3a8bdc45426b9d14abca7ed52c1a 2023-03-08 18:54:10,969 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 8336865ae81041d039911221d7ad790d/family1 of 8336865ae81041d039911221d7ad790d into cdeb3a8bdc45426b9d14abca7ed52c1a(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:54:10,969 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:10,970 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678301650505; duration=0sec 2023-03-08 18:54:10,970 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:10,991 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/6d7a92498d4847babd456d62ad2092eb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/081f8d562e674526a8ab8f92dab93080, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/0b5558a0a5b94b14a281684e168ca3d0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7af813807fbb4b038a5fcd83ab0df741, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/3ded35621e8a4cb38a47c6fb47fc05f7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/c53861a2a0ad46aabab7365114610ea4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016af30a77df4404b4b350de662d2c92, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/db06713932a742188a662b227d287b38, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/ae348ee094a84754b1fe7a5de5417383, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_] to archive 2023-03-08 18:54:10,992 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:10,994 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/6d7a92498d4847babd456d62ad2092eb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/6d7a92498d4847babd456d62ad2092eb 2023-03-08 18:54:10,995 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/081f8d562e674526a8ab8f92dab93080 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/081f8d562e674526a8ab8f92dab93080 2023-03-08 18:54:10,997 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/0b5558a0a5b94b14a281684e168ca3d0 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/0b5558a0a5b94b14a281684e168ca3d0 2023-03-08 18:54:10,998 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7af813807fbb4b038a5fcd83ab0df741 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7af813807fbb4b038a5fcd83ab0df741 2023-03-08 18:54:10,999 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/3ded35621e8a4cb38a47c6fb47fc05f7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/3ded35621e8a4cb38a47c6fb47fc05f7 2023-03-08 18:54:11,000 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/c53861a2a0ad46aabab7365114610ea4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/c53861a2a0ad46aabab7365114610ea4 2023-03-08 18:54:11,002 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016af30a77df4404b4b350de662d2c92 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016af30a77df4404b4b350de662d2c92 2023-03-08 18:54:11,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/db06713932a742188a662b227d287b38 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/db06713932a742188a662b227d287b38 2023-03-08 18:54:11,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/ae348ee094a84754b1fe7a5de5417383 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/ae348ee094a84754b1fe7a5de5417383 2023-03-08 18:54:11,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/2a30967a8bb244cc8ab46695c0d59c50_SeqId_44_ 2023-03-08 18:54:11,369 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512315f6494233e7644a7af32777b5ffccd44 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512315f6494233e7644a7af32777b5ffccd44 2023-03-08 18:54:11,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:54:11,396 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/802adfcb489443588e4a7cfc353437b8 first=Optional[r1] last=Optional[r4] 2023-03-08 18:54:11,397 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/802adfcb489443588e4a7cfc353437b8}] 2023-03-08 18:54:11,399 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/802adfcb489443588e4a7cfc353437b8 for inclusion in 8336865ae81041d039911221d7ad790d/family1 2023-03-08 18:54:11,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-08 18:54:11,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:11,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:11,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/802adfcb489443588e4a7cfc353437b8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2sf1atbnn0j9sdkr9mct3jdrno2c8o2aje44ovmpsbnkd16tthi87pglpork475g/family1/802adfcb489443588e4a7cfc353437b8 2023-03-08 18:54:11,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2sf1atbnn0j9sdkr9mct3jdrno2c8o2aje44ovmpsbnkd16tthi87pglpork475g/family1/802adfcb489443588e4a7cfc353437b8 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016c3b177c2c48478adcaf5d20847f7f_SeqId_47_ 2023-03-08 18:54:11,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2sf1atbnn0j9sdkr9mct3jdrno2c8o2aje44ovmpsbnkd16tthi87pglpork475g/family1/802adfcb489443588e4a7cfc353437b8 into 8336865ae81041d039911221d7ad790d/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016c3b177c2c48478adcaf5d20847f7f_SeqId_47_ - updating store file list. 2023-03-08 18:54:11,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016c3b177c2c48478adcaf5d20847f7f_SeqId_47_ into 8336865ae81041d039911221d7ad790d/family1 2023-03-08 18:54:11,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2sf1atbnn0j9sdkr9mct3jdrno2c8o2aje44ovmpsbnkd16tthi87pglpork475g/family1/802adfcb489443588e4a7cfc353437b8 into 8336865ae81041d039911221d7ad790d/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016c3b177c2c48478adcaf5d20847f7f_SeqId_47_) 2023-03-08 18:54:11,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2sf1atbnn0j9sdkr9mct3jdrno2c8o2aje44ovmpsbnkd16tthi87pglpork475g/family1/802adfcb489443588e4a7cfc353437b8 2023-03-08 18:54:11,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 18:54:11,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 18:54:11,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): 8336865ae81041d039911221d7ad790d/family1 is initiating minor compaction (all files) 2023-03-08 18:54:11,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2c2f10e9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:11,415 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8336865ae81041d039911221d7ad790d/family1 in testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:54:11,415 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7246705f57b8489ab3c8fa6fd877fa02, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/cdeb3a8bdc45426b9d14abca7ed52c1a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016c3b177c2c48478adcaf5d20847f7f_SeqId_47_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp, totalSize=178.1 K 2023-03-08 18:54:11,415 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 7246705f57b8489ab3c8fa6fd877fa02, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845940 2023-03-08 18:54:11,416 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting cdeb3a8bdc45426b9d14abca7ed52c1a, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245940 2023-03-08 18:54:11,416 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 016c3b177c2c48478adcaf5d20847f7f_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965245940 2023-03-08 18:54:11,417 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:54:11,418 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:11,420 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512018e0be4093bd64f7db42fda32fdecab95 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512018e0be4093bd64f7db42fda32fdecab95 2023-03-08 18:54:11,421 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120532f36b7a77c243dc9fc00e8d9577286c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120532f36b7a77c243dc9fc00e8d9577286c 2023-03-08 18:54:11,423 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228bc5110cf15444606a27faf9d7d52d7fc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228bc5110cf15444606a27faf9d7d52d7fc 2023-03-08 18:54:11,424 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317fe5695344894640a98b35d093b5d9d4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317fe5695344894640a98b35d093b5d9d4 2023-03-08 18:54:11,424 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-08 18:54:11,424 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-08 18:54:11,425 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:54:11,425 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:54:11,437 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8336865ae81041d039911221d7ad790d#family1#compaction#23 average throughput is 15.56 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:12,276 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511306f33bd08c1c94350b8fc8ce92515c358 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511306f33bd08c1c94350b8fc8ce92515c358 2023-03-08 18:54:12,277 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:54:12,305 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/051cb9e399a4486dba296d5cc38bafd7 first=Optional[r00] last=Optional[r8] 2023-03-08 18:54:12,305 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/051cb9e399a4486dba296d5cc38bafd7}] 2023-03-08 18:54:12,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/051cb9e399a4486dba296d5cc38bafd7 for inclusion in 8336865ae81041d039911221d7ad790d/family1 2023-03-08 18:54:12,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-08 18:54:12,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:12,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:12,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/051cb9e399a4486dba296d5cc38bafd7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__15aefu7v0hg0spme4hhavh68rjnus3q0it4833b74sphes3kau8mbgfnrsgappgo/family1/051cb9e399a4486dba296d5cc38bafd7 2023-03-08 18:54:12,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__15aefu7v0hg0spme4hhavh68rjnus3q0it4833b74sphes3kau8mbgfnrsgappgo/family1/051cb9e399a4486dba296d5cc38bafd7 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/316ec3691d9a46ee815c3f19f10b4c62_SeqId_49_ 2023-03-08 18:54:12,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__15aefu7v0hg0spme4hhavh68rjnus3q0it4833b74sphes3kau8mbgfnrsgappgo/family1/051cb9e399a4486dba296d5cc38bafd7 into 8336865ae81041d039911221d7ad790d/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/316ec3691d9a46ee815c3f19f10b4c62_SeqId_49_ - updating store file list. 2023-03-08 18:54:12,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/316ec3691d9a46ee815c3f19f10b4c62_SeqId_49_ into 8336865ae81041d039911221d7ad790d/family1 2023-03-08 18:54:12,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__15aefu7v0hg0spme4hhavh68rjnus3q0it4833b74sphes3kau8mbgfnrsgappgo/family1/051cb9e399a4486dba296d5cc38bafd7 into 8336865ae81041d039911221d7ad790d/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/316ec3691d9a46ee815c3f19f10b4c62_SeqId_49_) 2023-03-08 18:54:12,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithMonthlyPolicy__15aefu7v0hg0spme4hhavh68rjnus3q0it4833b74sphes3kau8mbgfnrsgappgo/family1/051cb9e399a4486dba296d5cc38bafd7 2023-03-08 18:54:12,322 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:54:12,323 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:12,325 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112079b35b88fb9d4a829e89d8c50ecf6f0f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112079b35b88fb9d4a829e89d8c50ecf6f0f 2023-03-08 18:54:12,326 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511285a366342f3e146549d1b0ae77a4260a4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511285a366342f3e146549d1b0ae77a4260a4 2023-03-08 18:54:12,327 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511307a3dbca807a94cf0a98e988e0f643f19 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511307a3dbca807a94cf0a98e988e0f643f19 2023-03-08 18:54:12,327 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-08 18:54:12,327 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 18:54:12,328 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:54:12,328 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:54:12,328 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:54:12,328 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:54:12,328 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:12,329 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:12,331 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.7270 sec 2023-03-08 18:54:12,449 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=258 (was 248) Potentially hanging thread: hconnection-0x6d9ba191-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/cluster_8eed2209-15fd-42d6-1e88-8ecffcfc5d58/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60526 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60554 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60606 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d9ba191-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_703111363_15 at /127.0.0.1:60466 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60610 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60628 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d9ba191-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60644 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60646 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60518 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d9ba191-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60492 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d9ba191-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x0a2ebee3-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/941e7ab823d54ac1ba8ee0dde1b85fdd java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x0a2ebee3-SendThread(127.0.0.1:54098) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60582 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60442 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54098@0x0a2ebee3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1420081506.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d9ba191-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60560 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60510 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_703111363_15 at /127.0.0.1:60670 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d9ba191-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60666 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-232402728_15 at /127.0.0.1:60542 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/cluster_8eed2209-15fd-42d6-1e88-8ecffcfc5d58/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=613 (was 593) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=234 (was 242), ProcessCount=187 (was 187), AvailableMemoryMB=7614 (was 7052) - AvailableMemoryMB LEAK? - 2023-03-08 18:54:12,464 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=258, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=234, ProcessCount=187, AvailableMemoryMB=7613 2023-03-08 18:54:12,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 18:54:12,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:12,471 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:54:12,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-08 18:54:12,472 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:54:12,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 18:54:12,474 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:12,474 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467 empty. 2023-03-08 18:54:12,475 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:12,475 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-08 18:54:12,484 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 18:54:12,487 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => c33f16a1cf86167f9f09d4bdddce6467, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:54:12,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 18:54:12,667 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/.tmp/family1/941e7ab823d54ac1ba8ee0dde1b85fdd as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/941e7ab823d54ac1ba8ee0dde1b85fdd 2023-03-08 18:54:12,673 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 8336865ae81041d039911221d7ad790d/family1 of 8336865ae81041d039911221d7ad790d into 941e7ab823d54ac1ba8ee0dde1b85fdd(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:54:12,674 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:54:12,674 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678301651414; duration=1sec 2023-03-08 18:54:12,674 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:12,684 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7246705f57b8489ab3c8fa6fd877fa02, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/cdeb3a8bdc45426b9d14abca7ed52c1a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016c3b177c2c48478adcaf5d20847f7f_SeqId_47_] to archive 2023-03-08 18:54:12,685 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:12,687 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7246705f57b8489ab3c8fa6fd877fa02 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/7246705f57b8489ab3c8fa6fd877fa02 2023-03-08 18:54:12,688 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/cdeb3a8bdc45426b9d14abca7ed52c1a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/cdeb3a8bdc45426b9d14abca7ed52c1a 2023-03-08 18:54:12,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016c3b177c2c48478adcaf5d20847f7f_SeqId_47_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/family1/016c3b177c2c48478adcaf5d20847f7f_SeqId_47_ 2023-03-08 18:54:12,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 18:54:12,894 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:12,894 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing c33f16a1cf86167f9f09d4bdddce6467, disabling compactions & flushes 2023-03-08 18:54:12,894 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:12,895 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:12,895 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. after waiting 0 ms 2023-03-08 18:54:12,895 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:12,895 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:12,895 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:12,897 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:54:12,899 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678301652898"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301652898"}]},"ts":"1678301652898"} 2023-03-08 18:54:12,900 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:54:12,901 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:54:12,901 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301652901"}]},"ts":"1678301652901"} 2023-03-08 18:54:12,902 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-08 18:54:12,909 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c33f16a1cf86167f9f09d4bdddce6467, ASSIGN}] 2023-03-08 18:54:12,910 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c33f16a1cf86167f9f09d4bdddce6467, ASSIGN 2023-03-08 18:54:12,911 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c33f16a1cf86167f9f09d4bdddce6467, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:54:13,062 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=c33f16a1cf86167f9f09d4bdddce6467, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:13,063 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678301653062"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301653062"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301653062"}]},"ts":"1678301653062"} 2023-03-08 18:54:13,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure c33f16a1cf86167f9f09d4bdddce6467, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:13,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 18:54:13,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:13,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c33f16a1cf86167f9f09d4bdddce6467, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:54:13,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:13,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:13,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:13,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:13,222 INFO [StoreOpener-c33f16a1cf86167f9f09d4bdddce6467-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:13,223 DEBUG [StoreOpener-c33f16a1cf86167f9f09d4bdddce6467-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1 2023-03-08 18:54:13,223 DEBUG [StoreOpener-c33f16a1cf86167f9f09d4bdddce6467-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1 2023-03-08 18:54:13,224 INFO [StoreOpener-c33f16a1cf86167f9f09d4bdddce6467-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c33f16a1cf86167f9f09d4bdddce6467 columnFamilyName family1 2023-03-08 18:54:13,225 INFO [StoreOpener-c33f16a1cf86167f9f09d4bdddce6467-1] regionserver.HStore(310): Store=c33f16a1cf86167f9f09d4bdddce6467/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:13,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:13,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:13,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:54:13,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:54:13,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened c33f16a1cf86167f9f09d4bdddce6467; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74426734, jitterRate=0.10904476046562195}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:54:13,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:13,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467., pid=25, masterSystemTime=1678301653217 2023-03-08 18:54:13,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:13,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:13,234 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=c33f16a1cf86167f9f09d4bdddce6467, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:13,235 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678301653234"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301653234"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301653234"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301653234"}]},"ts":"1678301653234"} 2023-03-08 18:54:13,238 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-08 18:54:13,238 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure c33f16a1cf86167f9f09d4bdddce6467, server=jenkins-hbase6.apache.org,46397,1678301621627 in 171 msec 2023-03-08 18:54:13,240 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-08 18:54:13,240 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c33f16a1cf86167f9f09d4bdddce6467, ASSIGN in 329 msec 2023-03-08 18:54:13,240 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:54:13,241 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301653240"}]},"ts":"1678301653240"} 2023-03-08 18:54:13,242 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-08 18:54:13,245 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:54:13,246 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 777 msec 2023-03-08 18:54:13,339 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:13,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 18:54:13,577 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-08 18:54:13,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:13,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:13,603 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:13,603 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:13,603 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:13,603 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:13,604 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:13,604 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:13,605 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:13,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:13,605 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:13,606 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:13,606 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:13,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:13,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:13,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:13,606 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:13,607 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:13,607 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:13,609 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:13,609 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:13,609 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:13,609 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:13,610 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 18:54:14,035 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113090919249c2754f29bc771c7b3fe95724 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015113090919249c2754f29bc771c7b3fe95724 2023-03-08 18:54:14,439 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/92894679fa484ad29e17bc3575cb995e 2023-03-08 18:54:14,445 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/92894679fa484ad29e17bc3575cb995e as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/92894679fa484ad29e17bc3575cb995e 2023-03-08 18:54:14,450 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/92894679fa484ad29e17bc3575cb995e, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-08 18:54:14,451 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 841ms, sequenceid=5, compaction requested=false 2023-03-08 18:54:14,451 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:14,451 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:14,452 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:14,452 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:14,452 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,452 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:14,452 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:14,453 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,453 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:14,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:14,454 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,454 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:14,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:14,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:14,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:14,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:14,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bfdad08[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:14,455 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:14,455 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,456 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,457 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:14,457 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:14,457 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,457 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:14,457 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:14,459 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,459 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:14,459 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:14,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:14,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:14,459 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:14,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:14,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:14,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:14,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:14,461 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:14,461 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:14,461 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:14,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:14,461 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,461 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:14,463 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,463 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:14,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:14,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,463 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:14,463 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:14,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:14,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:14,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:14,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,464 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:14,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:14,467 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,467 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:14,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:14,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:14,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:14,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:14,467 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:14,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:14,568 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:14,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:14,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:14,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:14,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,575 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:14,575 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:14,575 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:14,575 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:14,575 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,575 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,577 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,577 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:14,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:14,577 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:14,577 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,579 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:14,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:14,579 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:14,579 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:14,579 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:14,579 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:14,579 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:14,579 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:14,579 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:14,579 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 18:54:14,993 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511284c15e24a33b14bac97fe1e422e4335ab to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511284c15e24a33b14bac97fe1e422e4335ab 2023-03-08 18:54:15,398 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/1b4f7280879245d79f14c2ee7be402fc 2023-03-08 18:54:15,404 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/1b4f7280879245d79f14c2ee7be402fc as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/1b4f7280879245d79f14c2ee7be402fc 2023-03-08 18:54:15,409 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/1b4f7280879245d79f14c2ee7be402fc, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 18:54:15,410 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 831ms, sequenceid=9, compaction requested=false 2023-03-08 18:54:15,410 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:15,410 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:15,410 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:15,410 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:15,410 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,410 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:15,410 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:15,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,412 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:15,412 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,412 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:15,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:15,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:15,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:15,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:15,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62c9b604[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:15,414 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:15,414 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,415 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,415 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:15,415 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:15,415 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:15,415 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,415 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:15,418 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,418 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:15,418 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:15,418 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:15,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:15,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:15,420 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:15,420 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:15,420 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:15,420 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,420 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:15,421 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,421 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:15,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,421 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:15,422 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,423 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,425 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,425 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:15,425 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:15,425 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:15,425 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:15,425 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:15,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:15,425 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:15,425 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,426 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:15,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:15,426 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,426 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:15,426 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,426 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,426 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,526 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:15,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:15,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:15,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:15,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,533 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:15,533 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:15,533 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:15,533 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:15,534 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,534 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,535 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:15,535 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:15,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:15,535 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:15,536 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:15,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,536 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:15,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,536 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,536 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:15,536 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:15,536 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:15,536 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:15,536 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:15,537 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:15,537 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:15,537 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:15,537 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:15,537 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:15,954 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512018177f671d94b4b7c95ed34e806d24231 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512018177f671d94b4b7c95ed34e806d24231 2023-03-08 18:54:16,360 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/84af263fa8884ab692b2646f6aa96ecc 2023-03-08 18:54:16,366 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/84af263fa8884ab692b2646f6aa96ecc as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/84af263fa8884ab692b2646f6aa96ecc 2023-03-08 18:54:16,370 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/84af263fa8884ab692b2646f6aa96ecc, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 18:54:16,371 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 834ms, sequenceid=13, compaction requested=true 2023-03-08 18:54:16,371 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:16,371 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:16,372 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:16,372 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:16,372 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,372 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:16,372 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:16,375 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,375 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:16,375 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,375 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:16,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:16,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:16,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@330ae7ce[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:16,376 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:16,376 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,377 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,378 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:16,378 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,378 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:16,378 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:16,378 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:16,379 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,379 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:16,380 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:16,380 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:16,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:16,382 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:16,382 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:16,382 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:16,382 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,382 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:16,384 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,384 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:16,384 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:16,384 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,385 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:16,393 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:16,393 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:16,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:16,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,393 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:16,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:16,394 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:16,394 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:16,494 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:16,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:16,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:16,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:16,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,501 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:16,502 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:16,502 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:16,502 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:16,502 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,502 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,504 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:16,504 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:16,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:16,504 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:16,504 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:16,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,504 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,505 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,505 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:16,505 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:16,505 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:16,505 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:16,505 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:16,505 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:16,505 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:16,505 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:16,505 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:16,506 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:16,921 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120531e34d3d6563415b9757edcd01c60f94 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120531e34d3d6563415b9757edcd01c60f94 2023-03-08 18:54:17,326 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/b333ec18c15f4508977d104f9d61b2ce 2023-03-08 18:54:17,332 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/b333ec18c15f4508977d104f9d61b2ce as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/b333ec18c15f4508977d104f9d61b2ce 2023-03-08 18:54:17,336 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/b333ec18c15f4508977d104f9d61b2ce, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 18:54:17,337 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 832ms, sequenceid=17, compaction requested=true 2023-03-08 18:54:17,337 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:17,337 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:17,337 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:17,337 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:17,337 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,337 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:17,338 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:17,339 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,339 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:17,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:17,339 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,339 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:17,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:17,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:17,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:17,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:17,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e40b0c0[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:17,341 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:17,341 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,343 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,343 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:17,343 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:17,343 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,343 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:17,343 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:17,344 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,344 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:17,344 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:17,345 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:17,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:17,347 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:17,347 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:17,347 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:17,347 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,347 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:17,348 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,348 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:17,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:17,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,348 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:17,349 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,350 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:17,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,352 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,352 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,352 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,352 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:17,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:17,352 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:17,352 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,352 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:17,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:17,353 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:17,353 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:17,353 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,353 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:17,353 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,353 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:17,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:17,453 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:17,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:17,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:17,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:17,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,460 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:17,460 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:17,460 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:17,460 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:17,460 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,460 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,463 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:17,463 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:17,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:17,463 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:17,463 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:17,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,464 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:17,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,464 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,464 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:17,464 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:17,464 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:17,464 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:17,464 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:17,464 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:17,465 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:17,465 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:17,465 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:17,465 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:17,878 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228488be3d6012f46f79a2b542bac390b9d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228488be3d6012f46f79a2b542bac390b9d 2023-03-08 18:54:18,241 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:54:18,241 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0a2ebee3 to 127.0.0.1:54098 2023-03-08 18:54:18,241 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:54:18,242 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:54:18,242 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0f95eb0f to 127.0.0.1:54098 2023-03-08 18:54:18,243 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:54:18,243 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:54:18,243 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x50864191 to 127.0.0.1:54098 2023-03-08 18:54:18,243 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:54:18,284 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/69f97511b26c461cb2de36d636639674 2023-03-08 18:54:18,288 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/69f97511b26c461cb2de36d636639674 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/69f97511b26c461cb2de36d636639674 2023-03-08 18:54:18,292 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/69f97511b26c461cb2de36d636639674, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 18:54:18,293 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 828ms, sequenceid=21, compaction requested=true 2023-03-08 18:54:18,293 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:18,293 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:18,293 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:18,293 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:18,293 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,294 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:18,294 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:18,296 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,296 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:18,297 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,297 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1863dded[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:18,298 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:18,298 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,300 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,300 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:18,300 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,300 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:18,300 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:18,300 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:18,301 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,301 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:18,301 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:18,301 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:18,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:18,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:18,303 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:18,303 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:18,303 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:18,303 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,303 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:18,305 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,305 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:18,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,305 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:18,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:18,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:18,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:18,305 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:18,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:18,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,307 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:18,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:18,310 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:18,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:18,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:18,310 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:18,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:18,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:18,411 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:18,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:18,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:18,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:18,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,417 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:18,417 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:18,417 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:18,417 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:18,418 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,418 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,419 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:18,419 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:18,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:18,420 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:18,420 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:18,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,420 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:18,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,420 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:18,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:18,420 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:18,421 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:18,421 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:18,421 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:18,421 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:18,421 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:18,421 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:18,421 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:18,770 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:54:18,835 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123177f1664f36024152b7bee7665ac9dab9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123177f1664f36024152b7bee7665ac9dab9 2023-03-08 18:54:19,240 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/40efc226e72e43829d61027d410fd539 2023-03-08 18:54:19,246 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/40efc226e72e43829d61027d410fd539 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/40efc226e72e43829d61027d410fd539 2023-03-08 18:54:19,250 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/40efc226e72e43829d61027d410fd539, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 18:54:19,251 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 830ms, sequenceid=25, compaction requested=true 2023-03-08 18:54:19,251 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:19,251 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:19,251 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:19,251 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:19,251 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,251 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:19,251 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:19,253 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,253 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:19,253 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,253 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:19,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:19,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:19,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37b53173[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:19,254 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:19,254 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,256 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,256 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:19,256 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:19,256 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:19,256 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,256 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:19,261 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,261 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:19,261 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:19,262 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:19,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:19,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:19,263 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:19,263 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:19,263 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:19,264 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,264 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:19,265 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,265 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:19,265 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:19,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:19,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,265 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:19,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:19,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:19,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:19,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,266 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:19,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:19,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,269 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:19,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:19,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:19,269 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,369 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:19,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:19,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:19,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:19,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,376 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:19,376 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:19,376 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:19,376 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:19,377 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,377 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,379 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:19,379 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:19,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:19,379 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:19,380 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:19,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,380 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:19,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,380 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,381 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:19,381 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:19,381 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:19,381 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:19,381 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:19,381 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:19,381 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:19,381 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:19,381 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:19,381 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:19,793 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101f0e671a9518a4f978a5b996f7076d64b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101f0e671a9518a4f978a5b996f7076d64b 2023-03-08 18:54:20,199 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/315408d562994a0b888e4259f6f60c53 2023-03-08 18:54:20,204 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/315408d562994a0b888e4259f6f60c53 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/315408d562994a0b888e4259f6f60c53 2023-03-08 18:54:20,208 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/315408d562994a0b888e4259f6f60c53, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 18:54:20,209 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 828ms, sequenceid=29, compaction requested=true 2023-03-08 18:54:20,209 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:20,209 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:20,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:20,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:20,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:20,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:20,211 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:20,211 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,211 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@536584dc[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:20,213 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:20,213 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,214 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,214 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:20,214 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:20,214 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:20,214 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,214 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:20,215 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,215 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:20,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:20,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:20,215 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:20,215 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:20,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:20,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:20,217 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:20,217 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:20,217 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:20,217 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,217 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:20,219 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,219 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,219 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:20,220 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,221 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:20,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,223 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,223 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,223 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:20,223 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:20,223 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:20,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:20,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:20,223 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:20,224 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,224 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,224 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,224 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:20,224 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:20,224 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:20,324 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:20,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:20,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:20,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:20,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,331 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:20,331 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:20,331 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:20,331 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:20,331 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,331 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,332 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,332 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:20,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:20,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:20,332 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:20,333 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:20,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:20,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:20,334 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:20,334 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:20,334 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:20,334 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:20,334 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:20,334 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:20,334 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:20,334 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:20,334 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:20,749 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103363f587fcb9e49e19b643af11566dab1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103363f587fcb9e49e19b643af11566dab1 2023-03-08 18:54:21,155 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/12e1b420b3a94a558cb1323884ef2c55 2023-03-08 18:54:21,162 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/12e1b420b3a94a558cb1323884ef2c55 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/12e1b420b3a94a558cb1323884ef2c55 2023-03-08 18:54:21,167 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/12e1b420b3a94a558cb1323884ef2c55, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 18:54:21,169 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 834ms, sequenceid=33, compaction requested=true 2023-03-08 18:54:21,169 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:21,169 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:21,169 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:21,169 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:21,169 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,169 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:21,169 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:21,171 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,171 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:21,171 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:21,171 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:21,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:21,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7808f8c0[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:21,173 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:21,173 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,175 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,175 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:21,175 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:21,175 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,175 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:21,175 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:21,176 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:21,177 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,177 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:21,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,177 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:21,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:21,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:21,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:21,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:21,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:21,179 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:21,179 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:21,179 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:21,179 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,179 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:21,181 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,181 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:21,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:21,181 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:21,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:21,181 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:21,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:21,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:21,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,183 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:21,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,185 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,185 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:21,185 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:21,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:21,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:21,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:21,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:21,185 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:21,185 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,186 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,186 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,186 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:21,186 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,186 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:21,186 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,186 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:21,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:21,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:21,286 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:21,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:21,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:21,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:21,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,293 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:21,293 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:21,293 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:21,294 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:21,294 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,294 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,296 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:21,296 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:21,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:21,297 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:21,297 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:21,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:21,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:21,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:21,298 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:21,298 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:21,298 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:21,298 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:21,298 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:21,298 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:21,298 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:21,298 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:21,712 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509074286a38bf06746af9a22e4816b8e4846 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509074286a38bf06746af9a22e4816b8e4846 2023-03-08 18:54:22,118 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/dc0ec43e81574a01806eae8bd1f849be 2023-03-08 18:54:22,124 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/dc0ec43e81574a01806eae8bd1f849be as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/dc0ec43e81574a01806eae8bd1f849be 2023-03-08 18:54:22,130 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/dc0ec43e81574a01806eae8bd1f849be, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 18:54:22,131 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 833ms, sequenceid=37, compaction requested=true 2023-03-08 18:54:22,131 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:22,131 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:22,131 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:22,132 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:22,132 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,132 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:22,132 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:22,133 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,134 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:22,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:22,134 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,134 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:22,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:22,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:22,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:22,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:22,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e82de99[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:22,136 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:22,136 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,140 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,140 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:22,140 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:22,140 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:22,142 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:22,142 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:22,142 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:22,142 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,142 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:22,142 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:22,142 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:22,142 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:22,142 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:22,142 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,142 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:22,144 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,144 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,144 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:22,145 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:22,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,146 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:22,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:22,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:22,151 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:22,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:22,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:22,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:22,151 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:22,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:22,151 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:22,151 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:22,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:22,251 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:22,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:22,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:22,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:22,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,260 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:22,260 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:22,260 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:22,261 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:22,261 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,261 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,263 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:22,263 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:22,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:22,263 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:22,264 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:22,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:22,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:22,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:22,265 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:22,265 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:22,265 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:22,265 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:22,265 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:22,265 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:22,265 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. started... 2023-03-08 18:54:22,266 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c33f16a1cf86167f9f09d4bdddce6467 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:22,688 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120829be39f20d84c668c0489c516d1c4ae to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120829be39f20d84c668c0489c516d1c4ae 2023-03-08 18:54:23,095 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/16dcc941ba9845da90d0df9f34188e2f 2023-03-08 18:54:23,101 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/16dcc941ba9845da90d0df9f34188e2f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/16dcc941ba9845da90d0df9f34188e2f 2023-03-08 18:54:23,113 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/16dcc941ba9845da90d0df9f34188e2f, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 18:54:23,114 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c33f16a1cf86167f9f09d4bdddce6467 in 848ms, sequenceid=41, compaction requested=true 2023-03-08 18:54:23,115 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:23,115 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:23,115 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:23,115 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:23,115 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,115 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:23,115 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:23,117 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,117 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:23,117 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,117 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:23,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:23,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:23,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:23,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:23,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60a25693[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:23,119 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:23,119 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,120 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,121 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:23,121 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:23,121 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:23,121 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,121 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:23,123 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:23,123 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:23,123 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,123 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:23,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:23,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:23,126 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:23,126 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:23,126 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:23,126 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,126 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:23,128 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,128 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:23,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,128 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:23,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:23,130 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:23,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:23,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:23,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,132 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:23,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:23,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 18:54:23,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:23,135 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:23,135 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:23,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:23,135 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:23,235 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:23,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:23,238 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-08 18:54:23,240 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:23,243 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:23,246 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:23,246 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302263246 2023-03-08 18:54:23,251 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 18:54:23,253 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-08 18:54:23,253 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:23,253 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x0831b315 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:54:23,267 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38161cab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:54:23,267 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,124 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f4cdc3cbd59544c5b45edbe5095bf047 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103f4cdc3cbd59544c5b45edbe5095bf047 2023-03-08 18:54:24,125 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:24,145 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:54:24,146 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:55172, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:54:24,155 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:54:24,157 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:38460, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:54:24,169 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5d52e3d628cc4796af194a0b7e1e786b first=Optional[r3] last=Optional[r6] 2023-03-08 18:54:24,169 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5d52e3d628cc4796af194a0b7e1e786b}] 2023-03-08 18:54:24,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5d52e3d628cc4796af194a0b7e1e786b for inclusion in c33f16a1cf86167f9f09d4bdddce6467/family1 2023-03-08 18:54:24,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 18:54:24,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:24,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:24,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5d52e3d628cc4796af194a0b7e1e786b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dce27gedkc9evlen1oi3abj8pnfe8pqq06if2kjhv6uim785h02hnde7hobvso9m/family1/5d52e3d628cc4796af194a0b7e1e786b 2023-03-08 18:54:24,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dce27gedkc9evlen1oi3abj8pnfe8pqq06if2kjhv6uim785h02hnde7hobvso9m/family1/5d52e3d628cc4796af194a0b7e1e786b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_ 2023-03-08 18:54:24,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dce27gedkc9evlen1oi3abj8pnfe8pqq06if2kjhv6uim785h02hnde7hobvso9m/family1/5d52e3d628cc4796af194a0b7e1e786b into c33f16a1cf86167f9f09d4bdddce6467/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_ - updating store file list. 2023-03-08 18:54:24,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_ into c33f16a1cf86167f9f09d4bdddce6467/family1 2023-03-08 18:54:24,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dce27gedkc9evlen1oi3abj8pnfe8pqq06if2kjhv6uim785h02hnde7hobvso9m/family1/5d52e3d628cc4796af194a0b7e1e786b into c33f16a1cf86167f9f09d4bdddce6467/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_) 2023-03-08 18:54:24,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dce27gedkc9evlen1oi3abj8pnfe8pqq06if2kjhv6uim785h02hnde7hobvso9m/family1/5d52e3d628cc4796af194a0b7e1e786b 2023-03-08 18:54:24,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 18:54:24,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 18:54:24,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): c33f16a1cf86167f9f09d4bdddce6467/family1 is initiating minor compaction 2023-03-08 18:54:24,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@74cc6812; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:24,194 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c33f16a1cf86167f9f09d4bdddce6467/family1 in testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:24,194 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/1b4f7280879245d79f14c2ee7be402fc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/84af263fa8884ab692b2646f6aa96ecc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/b333ec18c15f4508977d104f9d61b2ce, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/69f97511b26c461cb2de36d636639674, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/40efc226e72e43829d61027d410fd539, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/315408d562994a0b888e4259f6f60c53, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/12e1b420b3a94a558cb1323884ef2c55, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/dc0ec43e81574a01806eae8bd1f849be, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/16dcc941ba9845da90d0df9f34188e2f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp, totalSize=50.0 K 2023-03-08 18:54:24,195 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 1b4f7280879245d79f14c2ee7be402fc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 18:54:24,195 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 84af263fa8884ab692b2646f6aa96ecc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 18:54:24,195 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting b333ec18c15f4508977d104f9d61b2ce, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 18:54:24,196 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 69f97511b26c461cb2de36d636639674, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 18:54:24,196 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 40efc226e72e43829d61027d410fd539, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 18:54:24,196 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 315408d562994a0b888e4259f6f60c53, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 18:54:24,196 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 12e1b420b3a94a558cb1323884ef2c55, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 18:54:24,196 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting dc0ec43e81574a01806eae8bd1f849be, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 18:54:24,196 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 16dcc941ba9845da90d0df9f34188e2f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 18:54:24,196 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 18:54:24,200 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:24,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:24,204 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228488be3d6012f46f79a2b542bac390b9d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228488be3d6012f46f79a2b542bac390b9d 2023-03-08 18:54:24,206 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123177f1664f36024152b7bee7665ac9dab9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123177f1664f36024152b7bee7665ac9dab9 2023-03-08 18:54:24,207 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101f0e671a9518a4f978a5b996f7076d64b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101f0e671a9518a4f978a5b996f7076d64b 2023-03-08 18:54:24,209 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103363f587fcb9e49e19b643af11566dab1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103363f587fcb9e49e19b643af11566dab1 2023-03-08 18:54:24,209 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-08 18:54:24,209 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 18:54:24,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,291 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): c33f16a1cf86167f9f09d4bdddce6467#family1#compaction#34 average throughput is 0.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:24,690 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054a1c6ade478a470fab6e1d10d30ecb41 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512054a1c6ade478a470fab6e1d10d30ecb41 2023-03-08 18:54:24,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:24,714 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/58ed9b401dd6447b83a41ac063dca4f5 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/58ed9b401dd6447b83a41ac063dca4f5 2023-03-08 18:54:24,722 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in c33f16a1cf86167f9f09d4bdddce6467/family1 of c33f16a1cf86167f9f09d4bdddce6467 into 58ed9b401dd6447b83a41ac063dca4f5(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:54:24,722 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:24,722 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678301664194; duration=0sec 2023-03-08 18:54:24,722 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:24,723 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/dfd20580da0642aeaa69706c60849e50 first=Optional[r1] last=Optional[r2] 2023-03-08 18:54:24,724 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/dfd20580da0642aeaa69706c60849e50}] 2023-03-08 18:54:24,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/dfd20580da0642aeaa69706c60849e50 for inclusion in c33f16a1cf86167f9f09d4bdddce6467/family1 2023-03-08 18:54:24,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-08 18:54:24,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:24,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:24,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/dfd20580da0642aeaa69706c60849e50 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__r23ndm3a2j8dtea7phsiu0u8274jramnb0qorlnqtqldl1dm31rrnqq27cldk4et/family1/dfd20580da0642aeaa69706c60849e50 2023-03-08 18:54:24,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__r23ndm3a2j8dtea7phsiu0u8274jramnb0qorlnqtqldl1dm31rrnqq27cldk4et/family1/dfd20580da0642aeaa69706c60849e50 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_ 2023-03-08 18:54:24,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__r23ndm3a2j8dtea7phsiu0u8274jramnb0qorlnqtqldl1dm31rrnqq27cldk4et/family1/dfd20580da0642aeaa69706c60849e50 into c33f16a1cf86167f9f09d4bdddce6467/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_ - updating store file list. 2023-03-08 18:54:24,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_ into c33f16a1cf86167f9f09d4bdddce6467/family1 2023-03-08 18:54:24,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__r23ndm3a2j8dtea7phsiu0u8274jramnb0qorlnqtqldl1dm31rrnqq27cldk4et/family1/dfd20580da0642aeaa69706c60849e50 into c33f16a1cf86167f9f09d4bdddce6467/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_) 2023-03-08 18:54:24,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithWeeklyPolicy__r23ndm3a2j8dtea7phsiu0u8274jramnb0qorlnqtqldl1dm31rrnqq27cldk4et/family1/dfd20580da0642aeaa69706c60849e50 2023-03-08 18:54:24,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 18:54:24,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 18:54:24,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): c33f16a1cf86167f9f09d4bdddce6467/family1 is initiating minor compaction (all files) 2023-03-08 18:54:24,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@609869a2; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:24,742 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c33f16a1cf86167f9f09d4bdddce6467/family1 in testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:54:24,742 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/92894679fa484ad29e17bc3575cb995e, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/58ed9b401dd6447b83a41ac063dca4f5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp, totalSize=176.8 K 2023-03-08 18:54:24,743 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 92894679fa484ad29e17bc3575cb995e, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845940 2023-03-08 18:54:24,743 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 58ed9b401dd6447b83a41ac063dca4f5, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245940 2023-03-08 18:54:24,744 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965245940 2023-03-08 18:54:24,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:24,745 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:24,747 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512018177f671d94b4b7c95ed34e806d24231 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512018177f671d94b4b7c95ed34e806d24231 2023-03-08 18:54:24,748 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120531e34d3d6563415b9757edcd01c60f94 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120531e34d3d6563415b9757edcd01c60f94 2023-03-08 18:54:24,748 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-08 18:54:24,748 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 18:54:24,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:24,749 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:24,749 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:24,750 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:24,752 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.5110 sec 2023-03-08 18:54:24,768 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): c33f16a1cf86167f9f09d4bdddce6467#family1#compaction#35 average throughput is 12.89 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:24,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/1b4f7280879245d79f14c2ee7be402fc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/84af263fa8884ab692b2646f6aa96ecc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/b333ec18c15f4508977d104f9d61b2ce, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/69f97511b26c461cb2de36d636639674, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/40efc226e72e43829d61027d410fd539, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/315408d562994a0b888e4259f6f60c53, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/12e1b420b3a94a558cb1323884ef2c55, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/dc0ec43e81574a01806eae8bd1f849be, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/16dcc941ba9845da90d0df9f34188e2f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_] to archive 2023-03-08 18:54:24,793 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:24,797 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/1b4f7280879245d79f14c2ee7be402fc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/1b4f7280879245d79f14c2ee7be402fc 2023-03-08 18:54:24,801 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/84af263fa8884ab692b2646f6aa96ecc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/84af263fa8884ab692b2646f6aa96ecc 2023-03-08 18:54:24,802 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/b333ec18c15f4508977d104f9d61b2ce to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/b333ec18c15f4508977d104f9d61b2ce 2023-03-08 18:54:24,805 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/69f97511b26c461cb2de36d636639674 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/69f97511b26c461cb2de36d636639674 2023-03-08 18:54:24,806 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/40efc226e72e43829d61027d410fd539 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/40efc226e72e43829d61027d410fd539 2023-03-08 18:54:24,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/315408d562994a0b888e4259f6f60c53 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/315408d562994a0b888e4259f6f60c53 2023-03-08 18:54:24,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/12e1b420b3a94a558cb1323884ef2c55 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/12e1b420b3a94a558cb1323884ef2c55 2023-03-08 18:54:24,810 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-03-08 18:54:24,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/dc0ec43e81574a01806eae8bd1f849be to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/dc0ec43e81574a01806eae8bd1f849be 2023-03-08 18:54:24,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/16dcc941ba9845da90d0df9f34188e2f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/16dcc941ba9845da90d0df9f34188e2f 2023-03-08 18:54:24,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/021fe1aaf9dd4a83b2166b90680eb3fc_SeqId_44_ 2023-03-08 18:54:24,878 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=254 (was 258), OpenFileDescriptor=596 (was 613), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=206 (was 234), ProcessCount=189 (was 187) - ProcessCount LEAK? -, AvailableMemoryMB=7202 (was 7613) 2023-03-08 18:54:24,890 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=254, OpenFileDescriptor=596, MaxFileDescriptor=60000, SystemLoadAverage=206, ProcessCount=189, AvailableMemoryMB=7202 2023-03-08 18:54:24,893 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 18:54:24,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:24,896 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:54:24,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-08 18:54:24,897 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:54:24,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 18:54:24,899 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:24,899 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:24,899 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:24,900 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1 empty. 2023-03-08 18:54:24,900 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1 empty. 2023-03-08 18:54:24,900 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21 empty. 2023-03-08 18:54:24,900 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:24,900 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:24,901 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:24,901 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-08 18:54:24,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 18:54:25,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 18:54:25,310 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-08 18:54:25,311 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 07707d9a61597d1f7ba6be12d211db21, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:54:25,312 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 40ac1522a08b03671fa59178494d47f1, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:54:25,312 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => a3c6eb6683fb37966ee22d85c9e40cc1, NAME => 'testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:54:25,328 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:25,328 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 07707d9a61597d1f7ba6be12d211db21, disabling compactions & flushes 2023-03-08 18:54:25,328 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:25,328 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:25,328 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. after waiting 0 ms 2023-03-08 18:54:25,328 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:25,328 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:25,328 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 07707d9a61597d1f7ba6be12d211db21: 2023-03-08 18:54:25,330 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:25,330 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 40ac1522a08b03671fa59178494d47f1, disabling compactions & flushes 2023-03-08 18:54:25,330 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:25,330 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:25,330 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. after waiting 0 ms 2023-03-08 18:54:25,330 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:25,330 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:25,330 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 40ac1522a08b03671fa59178494d47f1: 2023-03-08 18:54:25,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 18:54:25,616 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/.tmp/family1/462cf5a174454f2db51272dd87b33271 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/462cf5a174454f2db51272dd87b33271 2023-03-08 18:54:25,622 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c33f16a1cf86167f9f09d4bdddce6467/family1 of c33f16a1cf86167f9f09d4bdddce6467 into 462cf5a174454f2db51272dd87b33271(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:54:25,622 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:54:25,622 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1678301664742; duration=0sec 2023-03-08 18:54:25,622 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:25,682 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/92894679fa484ad29e17bc3575cb995e, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/58ed9b401dd6447b83a41ac063dca4f5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_] to archive 2023-03-08 18:54:25,683 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:25,685 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/92894679fa484ad29e17bc3575cb995e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/92894679fa484ad29e17bc3575cb995e 2023-03-08 18:54:25,686 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/58ed9b401dd6447b83a41ac063dca4f5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/58ed9b401dd6447b83a41ac063dca4f5 2023-03-08 18:54:25,687 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/family1/eecd0de0f1284ecab1bbd89c05b86b0d_SeqId_47_ 2023-03-08 18:54:25,726 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:25,726 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing a3c6eb6683fb37966ee22d85c9e40cc1, disabling compactions & flushes 2023-03-08 18:54:25,726 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:25,726 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:25,726 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. after waiting 0 ms 2023-03-08 18:54:25,726 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:25,726 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:25,726 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for a3c6eb6683fb37966ee22d85c9e40cc1: 2023-03-08 18:54:25,729 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:54:25,730 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678301665730"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301665730"}]},"ts":"1678301665730"} 2023-03-08 18:54:25,730 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678301665730"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301665730"}]},"ts":"1678301665730"} 2023-03-08 18:54:25,730 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678301665730"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301665730"}]},"ts":"1678301665730"} 2023-03-08 18:54:25,734 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 18:54:25,735 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:54:25,735 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301665735"}]},"ts":"1678301665735"} 2023-03-08 18:54:25,736 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-08 18:54:25,740 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a3c6eb6683fb37966ee22d85c9e40cc1, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=40ac1522a08b03671fa59178494d47f1, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=07707d9a61597d1f7ba6be12d211db21, ASSIGN}] 2023-03-08 18:54:25,742 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=07707d9a61597d1f7ba6be12d211db21, ASSIGN 2023-03-08 18:54:25,742 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a3c6eb6683fb37966ee22d85c9e40cc1, ASSIGN 2023-03-08 18:54:25,743 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=40ac1522a08b03671fa59178494d47f1, ASSIGN 2023-03-08 18:54:25,743 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=07707d9a61597d1f7ba6be12d211db21, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:54:25,743 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=40ac1522a08b03671fa59178494d47f1, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:54:25,744 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a3c6eb6683fb37966ee22d85c9e40cc1, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:54:25,895 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=07707d9a61597d1f7ba6be12d211db21, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:25,895 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=40ac1522a08b03671fa59178494d47f1, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:25,895 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=a3c6eb6683fb37966ee22d85c9e40cc1, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:25,895 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678301665895"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301665895"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301665895"}]},"ts":"1678301665895"} 2023-03-08 18:54:25,896 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678301665895"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301665895"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301665895"}]},"ts":"1678301665895"} 2023-03-08 18:54:25,896 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678301665895"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301665895"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301665895"}]},"ts":"1678301665895"} 2023-03-08 18:54:25,897 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure 07707d9a61597d1f7ba6be12d211db21, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:25,898 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=29, state=RUNNABLE; OpenRegionProcedure 40ac1522a08b03671fa59178494d47f1, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:25,899 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=28, state=RUNNABLE; OpenRegionProcedure a3c6eb6683fb37966ee22d85c9e40cc1, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:26,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 18:54:26,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:26,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 07707d9a61597d1f7ba6be12d211db21, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 18:54:26,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:26,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:26,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:26,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:26,055 INFO [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:26,057 DEBUG [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1 2023-03-08 18:54:26,057 DEBUG [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1 2023-03-08 18:54:26,057 INFO [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 07707d9a61597d1f7ba6be12d211db21 columnFamilyName family1 2023-03-08 18:54:26,059 INFO [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] regionserver.HStore(310): Store=07707d9a61597d1f7ba6be12d211db21/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:26,059 INFO [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:26,061 DEBUG [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family2 2023-03-08 18:54:26,061 DEBUG [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family2 2023-03-08 18:54:26,061 INFO [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 07707d9a61597d1f7ba6be12d211db21 columnFamilyName family2 2023-03-08 18:54:26,063 INFO [StoreOpener-07707d9a61597d1f7ba6be12d211db21-1] regionserver.HStore(310): Store=07707d9a61597d1f7ba6be12d211db21/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:26,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:26,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:26,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:54:26,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:54:26,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:54:26,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 07707d9a61597d1f7ba6be12d211db21; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71100844, jitterRate=0.05948513746261597}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:54:26,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 07707d9a61597d1f7ba6be12d211db21: 2023-03-08 18:54:26,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21., pid=31, masterSystemTime=1678301666050 2023-03-08 18:54:26,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:26,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:26,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:26,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 40ac1522a08b03671fa59178494d47f1, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 18:54:26,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:26,074 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=07707d9a61597d1f7ba6be12d211db21, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:26,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:26,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:26,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:26,074 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678301666073"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301666073"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301666073"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301666073"}]},"ts":"1678301666073"} 2023-03-08 18:54:26,075 INFO [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:26,077 DEBUG [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1/family1 2023-03-08 18:54:26,077 DEBUG [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1/family1 2023-03-08 18:54:26,078 INFO [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 40ac1522a08b03671fa59178494d47f1 columnFamilyName family1 2023-03-08 18:54:26,078 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-03-08 18:54:26,078 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure 07707d9a61597d1f7ba6be12d211db21, server=jenkins-hbase6.apache.org,46397,1678301621627 in 179 msec 2023-03-08 18:54:26,079 INFO [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] regionserver.HStore(310): Store=40ac1522a08b03671fa59178494d47f1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:26,079 INFO [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:26,080 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=07707d9a61597d1f7ba6be12d211db21, ASSIGN in 338 msec 2023-03-08 18:54:26,081 DEBUG [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1/family2 2023-03-08 18:54:26,081 DEBUG [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1/family2 2023-03-08 18:54:26,082 INFO [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 40ac1522a08b03671fa59178494d47f1 columnFamilyName family2 2023-03-08 18:54:26,083 INFO [StoreOpener-40ac1522a08b03671fa59178494d47f1-1] regionserver.HStore(310): Store=40ac1522a08b03671fa59178494d47f1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:26,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:26,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:26,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:54:26,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 40ac1522a08b03671fa59178494d47f1 2023-03-08 18:54:26,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:54:26,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 40ac1522a08b03671fa59178494d47f1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71415369, jitterRate=0.0641719251871109}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:54:26,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 40ac1522a08b03671fa59178494d47f1: 2023-03-08 18:54:26,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1., pid=32, masterSystemTime=1678301666050 2023-03-08 18:54:26,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:26,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:26,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:26,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a3c6eb6683fb37966ee22d85c9e40cc1, NAME => 'testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 18:54:26,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:26,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:26,093 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=40ac1522a08b03671fa59178494d47f1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:26,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:26,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:26,093 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678301666093"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301666093"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301666093"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301666093"}]},"ts":"1678301666093"} 2023-03-08 18:54:26,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=29 2023-03-08 18:54:26,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=29, state=SUCCESS; OpenRegionProcedure 40ac1522a08b03671fa59178494d47f1, server=jenkins-hbase6.apache.org,46397,1678301621627 in 197 msec 2023-03-08 18:54:26,099 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=40ac1522a08b03671fa59178494d47f1, ASSIGN in 357 msec 2023-03-08 18:54:26,099 INFO [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:26,101 DEBUG [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1/family1 2023-03-08 18:54:26,101 DEBUG [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1/family1 2023-03-08 18:54:26,101 INFO [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3c6eb6683fb37966ee22d85c9e40cc1 columnFamilyName family1 2023-03-08 18:54:26,103 INFO [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] regionserver.HStore(310): Store=a3c6eb6683fb37966ee22d85c9e40cc1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:26,103 INFO [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:26,104 DEBUG [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1/family2 2023-03-08 18:54:26,104 DEBUG [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1/family2 2023-03-08 18:54:26,105 INFO [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3c6eb6683fb37966ee22d85c9e40cc1 columnFamilyName family2 2023-03-08 18:54:26,106 INFO [StoreOpener-a3c6eb6683fb37966ee22d85c9e40cc1-1] regionserver.HStore(310): Store=a3c6eb6683fb37966ee22d85c9e40cc1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:26,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:26,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:26,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:54:26,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:54:26,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:54:26,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened a3c6eb6683fb37966ee22d85c9e40cc1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60387139, jitterRate=-0.10016150772571564}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:54:26,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for a3c6eb6683fb37966ee22d85c9e40cc1: 2023-03-08 18:54:26,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1., pid=33, masterSystemTime=1678301666050 2023-03-08 18:54:26,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:26,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:26,121 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=a3c6eb6683fb37966ee22d85c9e40cc1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:26,121 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678301666121"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301666121"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301666121"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301666121"}]},"ts":"1678301666121"} 2023-03-08 18:54:26,124 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=28 2023-03-08 18:54:26,124 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=28, state=SUCCESS; OpenRegionProcedure a3c6eb6683fb37966ee22d85c9e40cc1, server=jenkins-hbase6.apache.org,46397,1678301621627 in 223 msec 2023-03-08 18:54:26,126 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-03-08 18:54:26,126 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a3c6eb6683fb37966ee22d85c9e40cc1, ASSIGN in 384 msec 2023-03-08 18:54:26,127 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:54:26,127 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301666127"}]},"ts":"1678301666127"} 2023-03-08 18:54:26,128 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-08 18:54:26,131 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:54:26,132 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2380 sec 2023-03-08 18:54:27,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 18:54:27,003 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-08 18:54:27,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:27,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,012 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-08 18:54:27,012 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:27,012 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-08 18:54:27,012 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-08 18:54:27,012 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,012 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,014 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:27,014 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:27,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:27,014 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,014 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:27,014 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:27,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,015 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,015 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,015 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-08 18:54:27,015 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:27,015 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-08 18:54:27,015 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:27,015 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:27,016 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. started... 2023-03-08 18:54:27,015 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. started... 2023-03-08 18:54:27,016 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:54:27,016 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 40ac1522a08b03671fa59178494d47f1: 2023-03-08 18:54:27,016 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:27,016 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:27,016 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. started... 2023-03-08 18:54:27,016 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:27,016 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a3c6eb6683fb37966ee22d85c9e40cc1: 2023-03-08 18:54:27,017 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 07707d9a61597d1f7ba6be12d211db21 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-08 18:54:27,017 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:54:27,017 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:27,017 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:54:27,431 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303089eb88fa876d84118b9de45b35f2f22f2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303089eb88fa876d84118b9de45b35f2f22f2 2023-03-08 18:54:27,837 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/.tmp/family1/2b46f508cd4d4e12a18447425cb60a71 2023-03-08 18:54:27,843 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/.tmp/family1/2b46f508cd4d4e12a18447425cb60a71 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/2b46f508cd4d4e12a18447425cb60a71 2023-03-08 18:54:27,847 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/2b46f508cd4d4e12a18447425cb60a71, entries=1, sequenceid=5, filesize=4.9 K 2023-03-08 18:54:27,848 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 07707d9a61597d1f7ba6be12d211db21 in 831ms, sequenceid=5, compaction requested=false 2023-03-08 18:54:27,848 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-08 18:54:27,849 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 07707d9a61597d1f7ba6be12d211db21: 2023-03-08 18:54:27,849 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:27,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:54:27,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:54:27,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-08 18:54:27,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-08 18:54:27,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,852 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:27,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:27,853 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:27,853 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:27,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:27,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:27,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-08 18:54:27,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f9e6d3f[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:27,854 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-08 18:54:27,854 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,856 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,856 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:27,856 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-08 18:54:27,856 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-08 18:54:27,856 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:27,858 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,858 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-08 18:54:27,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,858 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:27,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:27,858 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-08 18:54:27,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-08 18:54:27,861 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-08 18:54:27,861 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:27,861 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:27,861 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,861 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:27,862 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,862 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:27,862 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:27,863 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:27,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,864 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:27,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,872 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,872 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:27,872 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,873 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:27,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:27,872 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:27,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:27,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-08 18:54:27,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:27,873 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,873 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,873 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,873 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,873 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:27,873 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:27,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:27,873 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:27,974 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:27,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:27,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:27,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:27,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,981 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-08 18:54:27,981 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:27,981 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-08 18:54:27,981 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-08 18:54:27,981 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,981 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,983 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:27,983 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:27,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:27,983 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:27,983 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:27,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:27,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-08 18:54:27,984 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:27,984 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-08 18:54:27,984 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:27,984 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-08 18:54:27,984 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:27,984 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. started... 2023-03-08 18:54:27,984 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:27,984 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a3c6eb6683fb37966ee22d85c9e40cc1: 2023-03-08 18:54:27,985 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:27,984 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:54:27,985 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:27,985 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. started... 2023-03-08 18:54:27,985 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. started... 2023-03-08 18:54:27,985 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:54:27,985 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 40ac1522a08b03671fa59178494d47f1: 2023-03-08 18:54:27,985 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:54:27,985 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:54:27,985 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 07707d9a61597d1f7ba6be12d211db21 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-08 18:54:27,985 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:54:28,399 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308abe18dba5038483495e4912c69ae85b1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308abe18dba5038483495e4912c69ae85b1 2023-03-08 18:54:28,805 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/.tmp/family1/ff72c4be6e044ecc87bd39a1e1a40953 2023-03-08 18:54:28,811 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/.tmp/family1/ff72c4be6e044ecc87bd39a1e1a40953 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/ff72c4be6e044ecc87bd39a1e1a40953 2023-03-08 18:54:28,815 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/ff72c4be6e044ecc87bd39a1e1a40953, entries=2, sequenceid=9, filesize=5.0 K 2023-03-08 18:54:28,816 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 07707d9a61597d1f7ba6be12d211db21 in 831ms, sequenceid=9, compaction requested=false 2023-03-08 18:54:28,816 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 07707d9a61597d1f7ba6be12d211db21: 2023-03-08 18:54:28,816 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:28,816 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:54:28,817 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:54:28,817 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,817 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-08 18:54:28,817 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-08 18:54:28,818 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,818 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:28,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:28,819 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,819 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:28,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:28,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:28,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:28,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-08 18:54:28,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20ef48f4[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:28,820 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-08 18:54:28,820 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,822 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,822 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-08 18:54:28,822 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,822 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-08 18:54:28,822 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:28,822 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:28,825 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-08 18:54:28,825 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,825 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:28,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:28,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:28,825 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-08 18:54:28,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:28,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:28,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:28,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:28,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-08 18:54:28,827 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-08 18:54:28,827 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:28,827 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:28,827 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,828 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:28,829 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,829 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:28,829 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,829 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:28,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:28,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:28,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:28,831 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:28,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,833 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,833 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:28,833 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:28,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:28,833 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-08 18:54:28,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:28,833 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:28,833 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:28,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-08 18:54:28,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:28,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:28,833 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:28,833 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:28,834 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:28,834 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,834 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,834 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,934 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:28,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:28,942 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-08 18:54:28,944 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 18:54:28,944 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 18:54:28,945 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:28,945 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302268945 2023-03-08 18:54:28,946 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:54:28,946 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-08 18:54:28,946 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 18:54:28,946 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x7185c703 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:54:28,950 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79fe7edc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:54:28,951 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-08 18:54:29,384 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303080d10e03aac7a4d48a02d32755dd4d6c7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303080d10e03aac7a4d48a02d32755dd4d6c7 2023-03-08 18:54:29,385 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 18:54:29,398 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:54:29,399 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:55176, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:54:29,407 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:54:29,409 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:38470, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:54:29,421 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/2f327f9a78f846b78d1af28ee2984a53 first=Optional[k0] last=Optional[k1] 2023-03-08 18:54:29,421 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/2f327f9a78f846b78d1af28ee2984a53}] 2023-03-08 18:54:29,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/2f327f9a78f846b78d1af28ee2984a53 for inclusion in 07707d9a61597d1f7ba6be12d211db21/family1 2023-03-08 18:54:29,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-08 18:54:29,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 18:54:29,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 07707d9a61597d1f7ba6be12d211db21: 2023-03-08 18:54:29,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/2f327f9a78f846b78d1af28ee2984a53 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testScannerOnBulkLoadRefHFiles__9umksu321rqu26ugb0p028lk4ikr7j9nkalu4d8s7i33pjt6g6c7cb9gv9anruur/family1/2f327f9a78f846b78d1af28ee2984a53 2023-03-08 18:54:29,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testScannerOnBulkLoadRefHFiles__9umksu321rqu26ugb0p028lk4ikr7j9nkalu4d8s7i33pjt6g6c7cb9gv9anruur/family1/2f327f9a78f846b78d1af28ee2984a53 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_ 2023-03-08 18:54:29,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testScannerOnBulkLoadRefHFiles__9umksu321rqu26ugb0p028lk4ikr7j9nkalu4d8s7i33pjt6g6c7cb9gv9anruur/family1/2f327f9a78f846b78d1af28ee2984a53 into 07707d9a61597d1f7ba6be12d211db21/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_ - updating store file list. 2023-03-08 18:54:29,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_ into 07707d9a61597d1f7ba6be12d211db21/family1 2023-03-08 18:54:29,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testScannerOnBulkLoadRefHFiles__9umksu321rqu26ugb0p028lk4ikr7j9nkalu4d8s7i33pjt6g6c7cb9gv9anruur/family1/2f327f9a78f846b78d1af28ee2984a53 into 07707d9a61597d1f7ba6be12d211db21/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_) 2023-03-08 18:54:29,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testScannerOnBulkLoadRefHFiles__9umksu321rqu26ugb0p028lk4ikr7j9nkalu4d8s7i33pjt6g6c7cb9gv9anruur/family1/2f327f9a78f846b78d1af28ee2984a53 2023-03-08 18:54:29,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 18:54:29,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 18:54:29,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): 07707d9a61597d1f7ba6be12d211db21/family1 is initiating minor compaction (all files) 2023-03-08 18:54:29,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@c30acd0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:29,440 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 07707d9a61597d1f7ba6be12d211db21/family1 in testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:54:29,440 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/2b46f508cd4d4e12a18447425cb60a71, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/ff72c4be6e044ecc87bd39a1e1a40953, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/.tmp, totalSize=15.1 K 2023-03-08 18:54:29,441 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 2b46f508cd4d4e12a18447425cb60a71, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301667003 2023-03-08 18:54:29,441 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting ff72c4be6e044ecc87bd39a1e1a40953, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301667003 2023-03-08 18:54:29,442 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301667003 2023-03-08 18:54:29,442 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 18:54:29,445 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:29,447 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303089eb88fa876d84118b9de45b35f2f22f2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303089eb88fa876d84118b9de45b35f2f22f2 2023-03-08 18:54:29,448 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308abe18dba5038483495e4912c69ae85b1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308abe18dba5038483495e4912c69ae85b1 2023-03-08 18:54:29,448 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-08 18:54:29,448 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 18:54:29,449 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-08 18:54:29,449 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 18:54:29,449 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:29,450 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 18:54:29,452 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 507 msec 2023-03-08 18:54:29,453 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 07707d9a61597d1f7ba6be12d211db21#family1#compaction#38 average throughput is 0.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:29,481 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=251 (was 254), OpenFileDescriptor=593 (was 596), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=197 (was 206), ProcessCount=186 (was 189), AvailableMemoryMB=7024 (was 7202) 2023-03-08 18:54:29,492 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=251, OpenFileDescriptor=593, MaxFileDescriptor=60000, SystemLoadAverage=197, ProcessCount=186, AvailableMemoryMB=7024 2023-03-08 18:54:29,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 18:54:29,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:29,496 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:54:29,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-08 18:54:29,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 18:54:29,497 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:54:29,499 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:29,499 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac empty. 2023-03-08 18:54:29,499 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:29,499 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-08 18:54:29,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 18:54:29,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 18:54:29,908 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 18:54:29,909 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => f9ff7d9a8ef2065b168b9dc4709311ac, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:54:30,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 18:54:30,315 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:30,315 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing f9ff7d9a8ef2065b168b9dc4709311ac, disabling compactions & flushes 2023-03-08 18:54:30,315 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:30,315 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:30,315 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. after waiting 0 ms 2023-03-08 18:54:30,315 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:30,315 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:30,315 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:30,318 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:54:30,318 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678301670318"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301670318"}]},"ts":"1678301670318"} 2023-03-08 18:54:30,320 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:54:30,321 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:54:30,321 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301670321"}]},"ts":"1678301670321"} 2023-03-08 18:54:30,322 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-08 18:54:30,325 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=f9ff7d9a8ef2065b168b9dc4709311ac, ASSIGN}] 2023-03-08 18:54:30,326 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=f9ff7d9a8ef2065b168b9dc4709311ac, ASSIGN 2023-03-08 18:54:30,327 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=f9ff7d9a8ef2065b168b9dc4709311ac, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:54:30,478 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=f9ff7d9a8ef2065b168b9dc4709311ac, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:30,478 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678301670478"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301670478"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301670478"}]},"ts":"1678301670478"} 2023-03-08 18:54:30,480 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure f9ff7d9a8ef2065b168b9dc4709311ac, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:30,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 18:54:30,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:30,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f9ff7d9a8ef2065b168b9dc4709311ac, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:54:30,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:30,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:30,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:30,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:30,637 INFO [StoreOpener-f9ff7d9a8ef2065b168b9dc4709311ac-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:30,638 DEBUG [StoreOpener-f9ff7d9a8ef2065b168b9dc4709311ac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1 2023-03-08 18:54:30,638 DEBUG [StoreOpener-f9ff7d9a8ef2065b168b9dc4709311ac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1 2023-03-08 18:54:30,639 INFO [StoreOpener-f9ff7d9a8ef2065b168b9dc4709311ac-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9ff7d9a8ef2065b168b9dc4709311ac columnFamilyName family1 2023-03-08 18:54:30,640 INFO [StoreOpener-f9ff7d9a8ef2065b168b9dc4709311ac-1] regionserver.HStore(310): Store=f9ff7d9a8ef2065b168b9dc4709311ac/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:30,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:30,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:30,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:54:30,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:54:30,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened f9ff7d9a8ef2065b168b9dc4709311ac; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66861198, jitterRate=-0.0036905109882354736}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:54:30,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:30,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac., pid=37, masterSystemTime=1678301670632 2023-03-08 18:54:30,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:30,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:30,649 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=f9ff7d9a8ef2065b168b9dc4709311ac, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:30,650 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678301670649"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301670649"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301670649"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301670649"}]},"ts":"1678301670649"} 2023-03-08 18:54:30,652 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-08 18:54:30,652 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure f9ff7d9a8ef2065b168b9dc4709311ac, server=jenkins-hbase6.apache.org,46397,1678301621627 in 171 msec 2023-03-08 18:54:30,654 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-08 18:54:30,654 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=f9ff7d9a8ef2065b168b9dc4709311ac, ASSIGN in 327 msec 2023-03-08 18:54:30,654 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:54:30,655 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301670654"}]},"ts":"1678301670654"} 2023-03-08 18:54:30,655 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-08 18:54:30,658 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:54:30,660 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1640 sec 2023-03-08 18:54:30,676 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/.tmp/family1/85f87101b39946a39166f68fa748a7da as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/85f87101b39946a39166f68fa748a7da 2023-03-08 18:54:30,680 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 07707d9a61597d1f7ba6be12d211db21/family1 of 07707d9a61597d1f7ba6be12d211db21 into 85f87101b39946a39166f68fa748a7da(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:54:30,680 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 07707d9a61597d1f7ba6be12d211db21: 2023-03-08 18:54:30,681 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1678301669440; duration=1sec 2023-03-08 18:54:30,681 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:30,786 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/2b46f508cd4d4e12a18447425cb60a71, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/ff72c4be6e044ecc87bd39a1e1a40953, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_] to archive 2023-03-08 18:54:30,787 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:30,789 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/2b46f508cd4d4e12a18447425cb60a71 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/2b46f508cd4d4e12a18447425cb60a71 2023-03-08 18:54:30,791 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/ff72c4be6e044ecc87bd39a1e1a40953 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/ff72c4be6e044ecc87bd39a1e1a40953 2023-03-08 18:54:30,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/family1/5f43e9d8b1794b02912073dc494f2b3d_SeqId_12_ 2023-03-08 18:54:31,580 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:54:31,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 18:54:31,604 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-08 18:54:31,611 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:31,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:31,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:31,636 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:31,636 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:31,636 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:31,636 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:31,637 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:31,637 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:31,639 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:31,639 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:31,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:31,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:31,639 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:31,639 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:31,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:31,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:31,640 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:31,640 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:31,640 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:31,640 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:31,640 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:31,641 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:31,641 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:31,641 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:31,641 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 18:54:32,065 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130aaa232465e6a4219b4a9ef6334b51ff3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130aaa232465e6a4219b4a9ef6334b51ff3 2023-03-08 18:54:32,469 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/9b3f3e35f0f846f5b977f875667d760b 2023-03-08 18:54:32,474 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/9b3f3e35f0f846f5b977f875667d760b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/9b3f3e35f0f846f5b977f875667d760b 2023-03-08 18:54:32,479 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/9b3f3e35f0f846f5b977f875667d760b, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-08 18:54:32,480 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 839ms, sequenceid=5, compaction requested=false 2023-03-08 18:54:32,481 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:32,481 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:32,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:32,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:32,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:32,481 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:32,483 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,483 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:32,483 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,483 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:32,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:32,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:32,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:32,485 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48712c66[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:32,485 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,486 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,487 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:32,487 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:32,487 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:32,487 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,487 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:32,489 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:32,489 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,489 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:32,489 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:32,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:32,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:32,491 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:32,491 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:32,491 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:32,491 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,491 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:32,494 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,494 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:32,494 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:32,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:32,494 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:32,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:32,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,495 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:32,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:32,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:32,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:32,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:32,498 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,498 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:32,599 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:32,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:32,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:32,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:32,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,606 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:32,606 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:32,606 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:32,606 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:32,607 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,607 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,608 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,608 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:32,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:32,608 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:32,608 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,609 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:32,609 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:32,610 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:32,610 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:32,610 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:32,610 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:32,610 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:32,610 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:32,610 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:32,610 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 18:54:33,025 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128911da16d5e794c22bece2950deb005e1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128911da16d5e794c22bece2950deb005e1 2023-03-08 18:54:33,031 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/326848cc892841f3b52b7a94f1a5cd50 2023-03-08 18:54:33,036 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/326848cc892841f3b52b7a94f1a5cd50 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/326848cc892841f3b52b7a94f1a5cd50 2023-03-08 18:54:33,040 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/326848cc892841f3b52b7a94f1a5cd50, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 18:54:33,041 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 431ms, sequenceid=9, compaction requested=false 2023-03-08 18:54:33,041 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:33,041 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:33,041 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:33,041 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:33,041 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,041 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:33,041 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:33,043 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,043 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:33,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:33,043 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,043 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:33,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:33,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:33,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:33,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:33,045 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:33,045 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a95ed35[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:33,049 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,049 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:33,049 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:33,049 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:33,049 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,049 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:33,051 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:33,051 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,051 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:33,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:33,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:33,051 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:33,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:33,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:33,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:33,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:33,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:33,053 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:33,053 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:33,053 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:33,053 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,053 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:33,055 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,055 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,055 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:33,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:33,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:33,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,056 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:33,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:33,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:33,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:33,057 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:33,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:33,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:33,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:33,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:33,059 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:33,059 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,060 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,060 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:33,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:33,160 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:33,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:33,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:33,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:33,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,166 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:33,167 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:33,167 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:33,167 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:33,167 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,167 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,170 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:33,170 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:33,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:33,170 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:33,170 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:33,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:33,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,171 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,171 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:33,171 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:33,171 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:33,171 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:33,171 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:33,173 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:33,173 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:33,173 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:33,173 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:33,174 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:33,586 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201a28c78e388cc49db9382fa6ba15a1f79 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201a28c78e388cc49db9382fa6ba15a1f79 2023-03-08 18:54:33,992 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/b1e309dcb491457a930008b1e3bc763b 2023-03-08 18:54:33,998 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/b1e309dcb491457a930008b1e3bc763b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b1e309dcb491457a930008b1e3bc763b 2023-03-08 18:54:34,002 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b1e309dcb491457a930008b1e3bc763b, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 18:54:34,003 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 829ms, sequenceid=13, compaction requested=true 2023-03-08 18:54:34,003 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:34,003 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:34,003 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:34,003 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:34,003 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,004 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:34,004 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:34,007 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:34,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:34,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:34,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:34,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:34,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:34,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:34,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@560e88cd[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:34,008 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:34,009 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,014 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,015 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:34,015 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:34,015 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:34,015 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,015 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:34,016 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:34,016 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,016 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:34,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:34,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:34,016 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:34,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:34,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:34,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:34,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:34,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:34,019 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:34,019 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:34,019 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:34,019 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,019 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:34,021 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,021 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,021 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:34,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:34,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:34,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:34,022 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:34,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:34,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:34,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,023 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:34,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,025 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,025 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,025 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,025 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,025 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:34,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:34,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:34,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:34,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:34,025 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:34,025 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:34,025 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:34,026 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,026 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:34,026 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,026 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:34,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:34,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:34,126 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:34,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:34,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:34,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:34,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,133 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:34,133 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:34,133 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:34,133 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:34,133 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,133 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,135 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,135 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:34,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:34,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:34,135 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,135 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:34,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,136 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:34,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,136 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,136 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:34,136 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,137 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:34,137 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:34,137 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:34,137 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:34,137 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:34,137 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:34,137 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:34,137 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:34,550 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120579addb7ea45e4a8fb7cad8723588d339 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120579addb7ea45e4a8fb7cad8723588d339 2023-03-08 18:54:34,955 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/7c48dd4f45c64d6aba236759da2733cc 2023-03-08 18:54:34,960 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/7c48dd4f45c64d6aba236759da2733cc as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/7c48dd4f45c64d6aba236759da2733cc 2023-03-08 18:54:34,964 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/7c48dd4f45c64d6aba236759da2733cc, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 18:54:34,965 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 828ms, sequenceid=17, compaction requested=true 2023-03-08 18:54:34,965 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:34,965 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:34,965 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:34,965 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:34,965 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,965 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:34,965 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:34,967 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,967 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:34,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:34,968 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,968 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:34,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:34,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cfcc210[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:34,969 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:34,969 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,971 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,971 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:34,971 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,971 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:34,971 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:34,971 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:34,973 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:34,973 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,973 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:34,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:34,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:34,973 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:34,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:34,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:34,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:34,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:34,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:34,975 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:34,975 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:34,975 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:34,975 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,975 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:34,977 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,977 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,977 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:34,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:34,977 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:34,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:34,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,978 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:34,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,981 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:34,981 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,981 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:34,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:34,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:34,981 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:34,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:34,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:34,981 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:34,982 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:34,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:34,982 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:34,982 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:34,982 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:34,982 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,982 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:34,982 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,082 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:35,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:35,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:35,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:35,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,089 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:35,089 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:35,089 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:35,089 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:35,090 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,090 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,092 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,092 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:35,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:35,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:35,092 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,092 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:35,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:35,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:35,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,093 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:35,093 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:35,094 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:35,094 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:35,094 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:35,094 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:35,094 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:35,094 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:35,506 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228abce98cb912c4a4b85689de2ed1cf3f4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228abce98cb912c4a4b85689de2ed1cf3f4 2023-03-08 18:54:35,912 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/530db117f6e844a8a1becaef8028458c 2023-03-08 18:54:35,917 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/530db117f6e844a8a1becaef8028458c as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/530db117f6e844a8a1becaef8028458c 2023-03-08 18:54:35,921 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/530db117f6e844a8a1becaef8028458c, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 18:54:35,922 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 828ms, sequenceid=21, compaction requested=true 2023-03-08 18:54:35,922 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:35,922 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:35,922 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:35,922 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:35,922 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,922 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:35,922 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:35,926 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,926 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:35,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:35,927 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,927 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:35,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:35,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:35,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:35,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:35,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4dafcdf5[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:35,928 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:35,928 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,930 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,930 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:35,930 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:35,930 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:35,930 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,930 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:35,932 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:35,932 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,932 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,932 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:35,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:35,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:35,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:35,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:35,934 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:35,934 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:35,934 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:35,934 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,934 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:35,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:35,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:35,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:35,936 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:35,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,937 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:35,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:35,939 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:35,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:35,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:35,939 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:35,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:35,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:35,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:35,940 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:35,940 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:35,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:35,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:36,040 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:36,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:36,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:36,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:36,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,047 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:36,047 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:36,047 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:36,047 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:36,048 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,048 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,049 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:36,049 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:36,049 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,049 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,050 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:36,050 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:36,050 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:36,051 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:36,051 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:36,051 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:36,051 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:36,051 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:36,463 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ba178f6262704329b584cbc0e49f5f5f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ba178f6262704329b584cbc0e49f5f5f 2023-03-08 18:54:36,869 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/e67e4df9e0f34cc69e8d4aa1fd7eb533 2023-03-08 18:54:36,874 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/e67e4df9e0f34cc69e8d4aa1fd7eb533 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/e67e4df9e0f34cc69e8d4aa1fd7eb533 2023-03-08 18:54:36,878 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/e67e4df9e0f34cc69e8d4aa1fd7eb533, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 18:54:36,879 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 828ms, sequenceid=25, compaction requested=true 2023-03-08 18:54:36,879 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:36,879 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:36,879 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:36,879 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:36,879 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,879 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:36,879 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:36,882 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,882 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:36,882 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,882 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:36,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:36,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:36,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:36,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58c67d30[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:36,884 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:36,884 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,885 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,885 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:36,885 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:36,885 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:36,885 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,886 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:36,887 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:36,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,887 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:36,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:36,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:36,887 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:36,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:36,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:36,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:36,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:36,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:36,889 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:36,889 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:36,889 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:36,889 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,889 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:36,890 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,890 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:36,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:36,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,890 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:36,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:36,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:36,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:36,891 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,892 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:36,896 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:36,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:36,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:36,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:36,896 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:36,997 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:36,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:37,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:37,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:37,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,003 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:37,003 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:37,003 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:37,003 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:37,004 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,004 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,005 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,005 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:37,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:37,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:37,005 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,005 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:37,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:37,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,006 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:37,006 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,006 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:37,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:37,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:37,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:37,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:37,007 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:37,007 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:37,007 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:37,419 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601013980a742f36f497d9bfe23e955d7c878 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601013980a742f36f497d9bfe23e955d7c878 2023-03-08 18:54:37,826 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/dbd2ba04381b46619e9f4334b3ef6334 2023-03-08 18:54:37,831 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/dbd2ba04381b46619e9f4334b3ef6334 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/dbd2ba04381b46619e9f4334b3ef6334 2023-03-08 18:54:37,835 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/dbd2ba04381b46619e9f4334b3ef6334, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 18:54:37,836 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 829ms, sequenceid=29, compaction requested=true 2023-03-08 18:54:37,836 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:37,836 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:37,836 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:37,836 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:37,836 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,836 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:37,836 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:37,838 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,838 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:37,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:37,838 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:37,838 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:37,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:37,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7aacce05[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:37,840 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:37,840 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,841 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,841 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:37,841 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:37,841 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:37,841 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,841 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:37,844 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,844 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:37,844 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:37,844 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:37,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:37,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:37,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:37,846 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:37,846 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:37,846 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:37,846 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,846 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:37,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:37,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:37,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:37,848 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,849 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:37,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:37,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:37,852 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:37,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:37,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:37,952 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:37,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:37,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:37,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:37,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,959 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:37,959 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:37,959 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:37,959 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:37,959 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,959 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,964 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:37,964 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:37,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:37,964 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:37,964 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:37,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,965 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:37,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,965 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,965 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:37,965 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:37,965 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:37,965 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:37,966 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:37,966 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:37,966 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:37,966 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:37,966 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:37,966 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:38,379 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037542c47f6b7e4602979af28033a13c99 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601037542c47f6b7e4602979af28033a13c99 2023-03-08 18:54:38,784 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/91307db8e4cb4c36b5066d4302a340cb 2023-03-08 18:54:38,790 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/91307db8e4cb4c36b5066d4302a340cb as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/91307db8e4cb4c36b5066d4302a340cb 2023-03-08 18:54:38,794 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/91307db8e4cb4c36b5066d4302a340cb, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 18:54:38,795 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 829ms, sequenceid=33, compaction requested=true 2023-03-08 18:54:38,795 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:38,795 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:38,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:38,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:38,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:38,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:38,797 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,797 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:38,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:38,798 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,798 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:38,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:38,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:38,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:38,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:38,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10aa7f02[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:38,799 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:38,799 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,800 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,801 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:38,801 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:38,801 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:38,801 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,801 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:38,802 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:38,802 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:38,802 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,802 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:38,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:38,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:38,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:38,805 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:38,805 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:38,805 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:38,805 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,805 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:38,807 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,807 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:38,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:38,807 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:38,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:38,808 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:38,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:38,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:38,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,809 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:38,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,811 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,811 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:38,811 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:38,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:38,811 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:38,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:38,811 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:38,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:38,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:38,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:38,811 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:38,812 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,812 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:38,812 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,812 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:38,812 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,812 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,912 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:38,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:38,917 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:38,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:38,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,919 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:38,919 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:38,919 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:38,919 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:38,920 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,920 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,921 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:38,921 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:38,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:38,922 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:38,922 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:38,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,922 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:38,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:38,923 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:38,923 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:38,923 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:38,923 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:38,937 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907c74833525bdf419aa1401e652bb79cd6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907c74833525bdf419aa1401e652bb79cd6 2023-03-08 18:54:39,348 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/cb9d8b8f24fc4c35b1060c8aab8372bc 2023-03-08 18:54:39,353 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/cb9d8b8f24fc4c35b1060c8aab8372bc as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/cb9d8b8f24fc4c35b1060c8aab8372bc 2023-03-08 18:54:39,357 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/cb9d8b8f24fc4c35b1060c8aab8372bc, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 18:54:39,358 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 435ms, sequenceid=37, compaction requested=true 2023-03-08 18:54:39,358 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:39,358 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:39,358 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:39,358 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:39,358 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,358 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:39,358 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:39,360 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,360 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:39,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:39,361 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,361 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:39,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:39,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:39,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:39,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:39,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2359d887[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:39,362 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:39,362 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,364 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,364 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:39,364 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:39,364 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,364 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:39,364 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:39,366 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,366 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:39,366 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:39,366 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:39,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:39,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:39,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:39,368 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:39,368 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:39,368 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:39,368 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,368 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:39,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,369 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:39,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:39,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:39,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:39,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:39,370 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:39,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:39,371 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:39,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,373 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,373 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:39,373 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:39,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:39,373 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:39,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:39,373 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:39,373 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:39,373 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:39,373 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:39,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:39,373 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:39,373 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:39,374 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,374 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,374 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,473 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:39,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:39,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:39,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:39,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,480 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:39,480 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:39,480 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 18:54:39,480 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 18:54:39,481 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,481 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,482 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:39,482 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:39,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:39,483 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,483 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:39,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:39,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,484 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 18:54:39,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,484 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 18:54:39,484 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:39,484 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 18:54:39,484 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:39,484 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:39,484 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:39,484 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. started... 2023-03-08 18:54:39,484 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9ff7d9a8ef2065b168b9dc4709311ac 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:39,498 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112016363b47c33f48a68ed4faf0248e5d8f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112016363b47c33f48a68ed4faf0248e5d8f 2023-03-08 18:54:39,909 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/b61cc1c1bbc1457a8e2ab73d9b929187 2023-03-08 18:54:39,914 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/b61cc1c1bbc1457a8e2ab73d9b929187 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b61cc1c1bbc1457a8e2ab73d9b929187 2023-03-08 18:54:39,918 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b61cc1c1bbc1457a8e2ab73d9b929187, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 18:54:39,919 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f9ff7d9a8ef2065b168b9dc4709311ac in 435ms, sequenceid=41, compaction requested=true 2023-03-08 18:54:39,919 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:39,919 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:39,919 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:39,919 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:39,919 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,919 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 18:54:39,919 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 18:54:39,920 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,920 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:39,921 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,921 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:39,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 18:54:39,922 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 18:54:39,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@489d0e21[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:39,922 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,923 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,923 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 18:54:39,923 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 18:54:39,923 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:39,923 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,924 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:39,925 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 18:54:39,925 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,925 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:39,925 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 18:54:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:39,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:54:39,927 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 18:54:39,927 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:39,927 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:39,927 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,927 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:39,929 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,929 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:39,930 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:39,930 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,931 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:39,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:39,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:39,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:39,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 18:54:39,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 18:54:39,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:39,936 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:39,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:39,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:39,936 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:39,937 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:39,937 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:39,937 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,937 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:39,937 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:40,037 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:40,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:40,039 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-08 18:54:40,041 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:40,042 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:40,043 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302280043 2023-03-08 18:54:40,043 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:40,044 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:54:40,045 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-08 18:54:40,045 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:40,046 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x06f092e5 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:54:40,050 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@cf20479, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:54:40,050 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,890 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f9ad50fcf16c48c9854ba4a2bca3dd3d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103f9ad50fcf16c48c9854ba4a2bca3dd3d 2023-03-08 18:54:40,892 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:40,906 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:54:40,907 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:41532, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:54:40,914 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:54:40,915 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:52092, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:54:40,930 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0ae6d59d1ea2416cb2736c4eacf167d6 first=Optional[r3] last=Optional[r6] 2023-03-08 18:54:40,931 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0ae6d59d1ea2416cb2736c4eacf167d6}] 2023-03-08 18:54:40,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0ae6d59d1ea2416cb2736c4eacf167d6 for inclusion in f9ff7d9a8ef2065b168b9dc4709311ac/family1 2023-03-08 18:54:40,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 18:54:40,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:40,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:40,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0ae6d59d1ea2416cb2736c4eacf167d6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__p9ldg8g59fmo7spu1b3bmo2q6a84tgbhv9rkebou9hshgtadgtjom1h2c6hs1e9t/family1/0ae6d59d1ea2416cb2736c4eacf167d6 2023-03-08 18:54:40,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__p9ldg8g59fmo7spu1b3bmo2q6a84tgbhv9rkebou9hshgtadgtjom1h2c6hs1e9t/family1/0ae6d59d1ea2416cb2736c4eacf167d6 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_ 2023-03-08 18:54:40,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__p9ldg8g59fmo7spu1b3bmo2q6a84tgbhv9rkebou9hshgtadgtjom1h2c6hs1e9t/family1/0ae6d59d1ea2416cb2736c4eacf167d6 into f9ff7d9a8ef2065b168b9dc4709311ac/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_ - updating store file list. 2023-03-08 18:54:40,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_ into f9ff7d9a8ef2065b168b9dc4709311ac/family1 2023-03-08 18:54:40,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__p9ldg8g59fmo7spu1b3bmo2q6a84tgbhv9rkebou9hshgtadgtjom1h2c6hs1e9t/family1/0ae6d59d1ea2416cb2736c4eacf167d6 into f9ff7d9a8ef2065b168b9dc4709311ac/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_) 2023-03-08 18:54:40,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__p9ldg8g59fmo7spu1b3bmo2q6a84tgbhv9rkebou9hshgtadgtjom1h2c6hs1e9t/family1/0ae6d59d1ea2416cb2736c4eacf167d6 2023-03-08 18:54:40,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 18:54:40,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 18:54:40,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): f9ff7d9a8ef2065b168b9dc4709311ac/family1 is initiating minor compaction 2023-03-08 18:54:40,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@e278130; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:40,951 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f9ff7d9a8ef2065b168b9dc4709311ac/family1 in testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:40,951 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/326848cc892841f3b52b7a94f1a5cd50, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b1e309dcb491457a930008b1e3bc763b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/7c48dd4f45c64d6aba236759da2733cc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/530db117f6e844a8a1becaef8028458c, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/e67e4df9e0f34cc69e8d4aa1fd7eb533, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/dbd2ba04381b46619e9f4334b3ef6334, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/91307db8e4cb4c36b5066d4302a340cb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/cb9d8b8f24fc4c35b1060c8aab8372bc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b61cc1c1bbc1457a8e2ab73d9b929187, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp, totalSize=50.0 K 2023-03-08 18:54:40,951 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 326848cc892841f3b52b7a94f1a5cd50, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 18:54:40,951 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting b1e309dcb491457a930008b1e3bc763b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 18:54:40,952 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 7c48dd4f45c64d6aba236759da2733cc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 18:54:40,952 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 530db117f6e844a8a1becaef8028458c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 18:54:40,952 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting e67e4df9e0f34cc69e8d4aa1fd7eb533, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 18:54:40,952 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting dbd2ba04381b46619e9f4334b3ef6334, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 18:54:40,952 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 91307db8e4cb4c36b5066d4302a340cb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 18:54:40,952 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting cb9d8b8f24fc4c35b1060c8aab8372bc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 18:54:40,952 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting b61cc1c1bbc1457a8e2ab73d9b929187, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 18:54:40,952 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 18:54:40,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:40,954 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:40,956 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228abce98cb912c4a4b85689de2ed1cf3f4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228abce98cb912c4a4b85689de2ed1cf3f4 2023-03-08 18:54:40,957 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ba178f6262704329b584cbc0e49f5f5f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ba178f6262704329b584cbc0e49f5f5f 2023-03-08 18:54:40,958 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601013980a742f36f497d9bfe23e955d7c878 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601013980a742f36f497d9bfe23e955d7c878 2023-03-08 18:54:40,959 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601037542c47f6b7e4602979af28033a13c99 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601037542c47f6b7e4602979af28033a13c99 2023-03-08 18:54:40,959 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-08 18:54:40,959 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 18:54:40,959 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,960 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,960 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,960 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,960 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,960 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,960 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,960 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:40,999 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): f9ff7d9a8ef2065b168b9dc4709311ac#family1#compaction#49 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:41,430 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055c034fbece0242c383d2503725467da5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512055c034fbece0242c383d2503725467da5 2023-03-08 18:54:41,431 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:41,436 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/f003e3293629479dbab9cd337388cecd as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/f003e3293629479dbab9cd337388cecd 2023-03-08 18:54:41,441 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f9ff7d9a8ef2065b168b9dc4709311ac/family1 of f9ff7d9a8ef2065b168b9dc4709311ac into f003e3293629479dbab9cd337388cecd(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:54:41,441 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:41,441 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678301680951; duration=0sec 2023-03-08 18:54:41,442 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:41,458 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cc234a33991d452cb9fdad2568d91b0f first=Optional[r00] last=Optional[r2] 2023-03-08 18:54:41,459 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cc234a33991d452cb9fdad2568d91b0f}] 2023-03-08 18:54:41,461 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cc234a33991d452cb9fdad2568d91b0f for inclusion in f9ff7d9a8ef2065b168b9dc4709311ac/family1 2023-03-08 18:54:41,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 18:54:41,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:41,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:41,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cc234a33991d452cb9fdad2568d91b0f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r0u2c1j7g98qefkuuq3jape3hiiujcg9rf844ddngmsvrkd7ht4qg0488bsagl1r/family1/cc234a33991d452cb9fdad2568d91b0f 2023-03-08 18:54:41,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r0u2c1j7g98qefkuuq3jape3hiiujcg9rf844ddngmsvrkd7ht4qg0488bsagl1r/family1/cc234a33991d452cb9fdad2568d91b0f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_ 2023-03-08 18:54:41,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r0u2c1j7g98qefkuuq3jape3hiiujcg9rf844ddngmsvrkd7ht4qg0488bsagl1r/family1/cc234a33991d452cb9fdad2568d91b0f into f9ff7d9a8ef2065b168b9dc4709311ac/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_ - updating store file list. 2023-03-08 18:54:41,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_ into f9ff7d9a8ef2065b168b9dc4709311ac/family1 2023-03-08 18:54:41,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r0u2c1j7g98qefkuuq3jape3hiiujcg9rf844ddngmsvrkd7ht4qg0488bsagl1r/family1/cc234a33991d452cb9fdad2568d91b0f into f9ff7d9a8ef2065b168b9dc4709311ac/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_) 2023-03-08 18:54:41,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r0u2c1j7g98qefkuuq3jape3hiiujcg9rf844ddngmsvrkd7ht4qg0488bsagl1r/family1/cc234a33991d452cb9fdad2568d91b0f 2023-03-08 18:54:41,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 18:54:41,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 18:54:41,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): f9ff7d9a8ef2065b168b9dc4709311ac/family1 is initiating minor compaction (all files) 2023-03-08 18:54:41,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6849dfb5; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:41,473 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f9ff7d9a8ef2065b168b9dc4709311ac/family1 in testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:54:41,473 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/9b3f3e35f0f846f5b977f875667d760b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/f003e3293629479dbab9cd337388cecd, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp, totalSize=337.0 K 2023-03-08 18:54:41,473 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 9b3f3e35f0f846f5b977f875667d760b, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845940 2023-03-08 18:54:41,474 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting f003e3293629479dbab9cd337388cecd, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245940 2023-03-08 18:54:41,474 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845940 2023-03-08 18:54:41,475 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:41,475 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:41,477 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130aaa232465e6a4219b4a9ef6334b51ff3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130aaa232465e6a4219b4a9ef6334b51ff3 2023-03-08 18:54:41,479 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201a28c78e388cc49db9382fa6ba15a1f79 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201a28c78e388cc49db9382fa6ba15a1f79 2023-03-08 18:54:41,486 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120579addb7ea45e4a8fb7cad8723588d339 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120579addb7ea45e4a8fb7cad8723588d339 2023-03-08 18:54:41,487 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-08 18:54:41,488 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 18:54:41,491 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/326848cc892841f3b52b7a94f1a5cd50, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b1e309dcb491457a930008b1e3bc763b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/7c48dd4f45c64d6aba236759da2733cc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/530db117f6e844a8a1becaef8028458c, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/e67e4df9e0f34cc69e8d4aa1fd7eb533, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/dbd2ba04381b46619e9f4334b3ef6334, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/91307db8e4cb4c36b5066d4302a340cb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/cb9d8b8f24fc4c35b1060c8aab8372bc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b61cc1c1bbc1457a8e2ab73d9b929187, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_] to archive 2023-03-08 18:54:41,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 18:54:41,492 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 18:54:41,492 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:41,493 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:54:41,493 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:41,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.4530 sec 2023-03-08 18:54:41,497 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/326848cc892841f3b52b7a94f1a5cd50 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/326848cc892841f3b52b7a94f1a5cd50 2023-03-08 18:54:41,499 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b1e309dcb491457a930008b1e3bc763b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b1e309dcb491457a930008b1e3bc763b 2023-03-08 18:54:41,500 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/7c48dd4f45c64d6aba236759da2733cc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/7c48dd4f45c64d6aba236759da2733cc 2023-03-08 18:54:41,502 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/530db117f6e844a8a1becaef8028458c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/530db117f6e844a8a1becaef8028458c 2023-03-08 18:54:41,503 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/e67e4df9e0f34cc69e8d4aa1fd7eb533 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/e67e4df9e0f34cc69e8d4aa1fd7eb533 2023-03-08 18:54:41,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/dbd2ba04381b46619e9f4334b3ef6334 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/dbd2ba04381b46619e9f4334b3ef6334 2023-03-08 18:54:41,506 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): f9ff7d9a8ef2065b168b9dc4709311ac#family1#compaction#50 average throughput is 12.89 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:41,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/91307db8e4cb4c36b5066d4302a340cb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/91307db8e4cb4c36b5066d4302a340cb 2023-03-08 18:54:41,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/cb9d8b8f24fc4c35b1060c8aab8372bc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/cb9d8b8f24fc4c35b1060c8aab8372bc 2023-03-08 18:54:41,519 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b61cc1c1bbc1457a8e2ab73d9b929187 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/b61cc1c1bbc1457a8e2ab73d9b929187 2023-03-08 18:54:41,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/43709f35fb564a5f93af6b1f601dc4d3_SeqId_44_ 2023-03-08 18:54:41,538 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/.tmp/family1/fcfe60ffcd8f44adb9a0cbbb68a4a0f6 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/fcfe60ffcd8f44adb9a0cbbb68a4a0f6 2023-03-08 18:54:41,545 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f9ff7d9a8ef2065b168b9dc4709311ac/family1 of f9ff7d9a8ef2065b168b9dc4709311ac into fcfe60ffcd8f44adb9a0cbbb68a4a0f6(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:54:41,545 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:54:41,546 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1678301681473; duration=0sec 2023-03-08 18:54:41,546 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:41,588 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/9b3f3e35f0f846f5b977f875667d760b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/f003e3293629479dbab9cd337388cecd, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_] to archive 2023-03-08 18:54:41,589 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:41,591 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/9b3f3e35f0f846f5b977f875667d760b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/9b3f3e35f0f846f5b977f875667d760b 2023-03-08 18:54:41,595 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/f003e3293629479dbab9cd337388cecd to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/f003e3293629479dbab9cd337388cecd 2023-03-08 18:54:41,596 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/family1/df6e7a09e0934ffa86a94e114c7858a3_SeqId_47_ 2023-03-08 18:54:41,609 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=253 (was 251) - Thread LEAK? -, OpenFileDescriptor=617 (was 593) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=198 (was 197) - SystemLoadAverage LEAK? -, ProcessCount=178 (was 186), AvailableMemoryMB=7121 (was 7024) - AvailableMemoryMB LEAK? - 2023-03-08 18:54:41,626 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=253, OpenFileDescriptor=617, MaxFileDescriptor=60000, SystemLoadAverage=198, ProcessCount=178, AvailableMemoryMB=7120 2023-03-08 18:54:41,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 18:54:41,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:41,630 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:54:41,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-08 18:54:41,630 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:54:41,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 18:54:41,632 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:41,633 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 empty. 2023-03-08 18:54:41,633 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:41,633 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-08 18:54:41,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 18:54:41,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 18:54:42,041 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-08 18:54:42,042 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 75fe5187a546ce484df6e7222327ef89, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:54:42,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 18:54:42,448 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:42,449 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 75fe5187a546ce484df6e7222327ef89, disabling compactions & flushes 2023-03-08 18:54:42,449 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:42,449 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:42,449 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. after waiting 0 ms 2023-03-08 18:54:42,449 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:42,449 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:42,449 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:42,452 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:54:42,453 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301682452"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301682452"}]},"ts":"1678301682452"} 2023-03-08 18:54:42,454 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:54:42,455 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:54:42,455 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301682455"}]},"ts":"1678301682455"} 2023-03-08 18:54:42,456 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-08 18:54:42,459 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, ASSIGN}] 2023-03-08 18:54:42,461 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, ASSIGN 2023-03-08 18:54:42,461 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:54:42,612 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:42,613 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301682612"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301682612"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301682612"}]},"ts":"1678301682612"} 2023-03-08 18:54:42,614 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:42,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 18:54:42,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:42,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75fe5187a546ce484df6e7222327ef89, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:54:42,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:42,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:42,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:42,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:42,771 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:42,772 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:42,772 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:42,772 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75fe5187a546ce484df6e7222327ef89 columnFamilyName family1 2023-03-08 18:54:42,774 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(310): Store=75fe5187a546ce484df6e7222327ef89/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:42,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:42,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:42,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:42,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:54:42,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 75fe5187a546ce484df6e7222327ef89; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71896168, jitterRate=0.07133638858795166}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:54:42,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:42,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89., pid=41, masterSystemTime=1678301682766 2023-03-08 18:54:42,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:42,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:42,783 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:42,783 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301682783"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301682783"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301682783"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301682783"}]},"ts":"1678301682783"} 2023-03-08 18:54:42,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-08 18:54:42,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627 in 170 msec 2023-03-08 18:54:42,787 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-08 18:54:42,787 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, ASSIGN in 326 msec 2023-03-08 18:54:42,788 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:54:42,788 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301682788"}]},"ts":"1678301682788"} 2023-03-08 18:54:42,789 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-08 18:54:42,791 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:54:42,792 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1630 sec 2023-03-08 18:54:43,370 DEBUG [regionserver/jenkins-hbase6:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 18:54:43,613 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:43,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 18:54:43,736 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-08 18:54:43,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:43,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:43,760 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:43,760 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:43,760 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:43,760 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:43,760 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:43,760 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:43,762 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:43,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:43,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:43,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:43,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:43,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:43,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:43,763 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:43,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:43,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:43,763 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:43,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:43,764 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:43,764 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:43,764 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:43,764 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:43,764 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:43,764 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:43,764 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:43,765 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 18:54:43,789 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130ec29e013383d48669fba4b4b86f8756e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130ec29e013383d48669fba4b4b86f8756e 2023-03-08 18:54:44,193 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/0738496e548e4e578ef18f1a3d685870 2023-03-08 18:54:44,198 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/0738496e548e4e578ef18f1a3d685870 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0738496e548e4e578ef18f1a3d685870 2023-03-08 18:54:44,202 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0738496e548e4e578ef18f1a3d685870, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-08 18:54:44,203 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 438ms, sequenceid=5, compaction requested=false 2023-03-08 18:54:44,204 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:44,204 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:44,204 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:44,204 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:44,204 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,204 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:44,204 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:44,206 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,206 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:44,206 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,206 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:44,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:44,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54dd2dd1[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:44,208 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:44,208 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,209 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:44,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:44,209 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,209 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:44,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:44,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,211 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:44,211 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:44,212 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:44,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:44,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:44,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:44,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:44,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:44,214 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:44,214 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:44,214 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:44,214 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,214 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:44,216 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,216 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:44,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,216 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:44,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:44,216 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:44,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,217 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:44,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:44,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:44,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 18:54:44,220 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:44,220 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:44,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:44,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:44,320 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:44,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:44,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:44,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:44,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,327 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:44,327 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:44,327 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:44,327 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:44,328 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,328 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,329 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,329 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:44,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:44,330 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,330 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,330 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,330 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,331 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:44,331 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,331 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:44,331 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:44,331 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:44,331 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:44,331 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:44,331 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:44,331 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:44,331 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 18:54:44,744 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ab7e7d196b1a43cdbc8b04a1aef0f421 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128ab7e7d196b1a43cdbc8b04a1aef0f421 2023-03-08 18:54:44,750 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/baa5198c024c4ac78178d4e66adf06a5 2023-03-08 18:54:44,755 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/baa5198c024c4ac78178d4e66adf06a5 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/baa5198c024c4ac78178d4e66adf06a5 2023-03-08 18:54:44,759 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/baa5198c024c4ac78178d4e66adf06a5, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 18:54:44,760 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 429ms, sequenceid=9, compaction requested=false 2023-03-08 18:54:44,760 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:44,760 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:44,760 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:44,760 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:44,760 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,760 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:44,760 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:44,762 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,762 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:44,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:44,762 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,762 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:44,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:44,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:44,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:44,764 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:44,764 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17cc1a27[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:44,764 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:44,764 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,765 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,765 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:44,765 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:44,765 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:44,765 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,765 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:44,767 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,767 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:44,767 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:44,767 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:44,769 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:44,769 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:44,770 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:44,770 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,770 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:44,772 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,772 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:44,772 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:44,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:44,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:44,772 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,773 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:44,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:44,783 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:44,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:44,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:44,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:44,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:44,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:44,783 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:44,884 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:44,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:44,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:44,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:44,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,890 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:44,890 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:44,890 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:44,890 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:44,891 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,891 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,892 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:44,892 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:44,892 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:44,892 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:44,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:44,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:44,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:44,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:44,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:44,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:44,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:44,894 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:44,894 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:44,894 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:45,307 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201beba1d3b171c4b789837cae82bf86180 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201beba1d3b171c4b789837cae82bf86180 2023-03-08 18:54:45,713 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/c1622c9ef63049c69045098049b51e64 2023-03-08 18:54:45,718 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/c1622c9ef63049c69045098049b51e64 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/c1622c9ef63049c69045098049b51e64 2023-03-08 18:54:45,721 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/c1622c9ef63049c69045098049b51e64, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 18:54:45,722 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 828ms, sequenceid=13, compaction requested=true 2023-03-08 18:54:45,722 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:45,722 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:45,722 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:45,722 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:45,722 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,723 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:45,723 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:45,724 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,724 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:45,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:45,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:45,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:45,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7abc65b9[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:45,726 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:45,726 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,727 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:45,728 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,728 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:45,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:45,728 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:45,729 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,729 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:45,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:45,729 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:45,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:45,729 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:45,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:45,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:45,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:45,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:45,731 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:45,731 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:45,731 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:45,731 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:45,731 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,731 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:45,734 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,734 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:45,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:45,734 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:45,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:45,734 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:45,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:45,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:45,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,735 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:45,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,737 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,737 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,737 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:45,737 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:45,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:45,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:45,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:45,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 18:54:45,738 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:45,737 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,738 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:45,737 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:45,738 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:45,738 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:45,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:45,738 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,738 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,838 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:45,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:45,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:45,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:45,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,845 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:45,845 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:45,846 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:45,846 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:45,846 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,846 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,847 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:45,847 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:45,848 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:45,848 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,849 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,849 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:45,849 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:45,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:45,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:45,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:45,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:45,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:45,849 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:45,849 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:45,849 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:46,265 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d50a8e14704e4037bd5703bc81671e33 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d50a8e14704e4037bd5703bc81671e33 2023-03-08 18:54:46,671 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/2d9e7471cb0141a29353e3b1ba3ffb4c 2023-03-08 18:54:46,676 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/2d9e7471cb0141a29353e3b1ba3ffb4c as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2d9e7471cb0141a29353e3b1ba3ffb4c 2023-03-08 18:54:46,680 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2d9e7471cb0141a29353e3b1ba3ffb4c, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 18:54:46,681 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 832ms, sequenceid=17, compaction requested=true 2023-03-08 18:54:46,681 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:46,681 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:46,681 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:46,681 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:46,681 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,681 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:46,682 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:46,683 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,683 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:46,684 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:46,684 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:46,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:46,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a7090c9[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:46,685 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:46,685 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,688 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,688 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:46,688 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,688 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:46,688 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:46,688 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:46,690 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:46,690 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,690 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:46,690 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:46,692 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:46,692 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:46,692 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:46,692 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,692 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:46,694 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,694 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:46,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,694 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:46,695 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:46,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:46,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,696 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:46,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:46,698 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,698 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:46,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:46,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 18:54:46,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:46,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:46,699 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:46,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:46,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:46,699 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:46,799 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:46,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:46,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:46,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:46,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,805 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:46,806 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:46,806 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:46,806 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:46,806 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,806 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,808 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,808 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:46,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:46,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:46,808 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:46,808 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:46,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:46,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:46,809 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:46,809 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:46,809 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:46,810 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:46,810 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:46,810 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:46,810 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:46,810 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:47,222 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122839c3a8fd54b44fc998c04a1e45f37176 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122839c3a8fd54b44fc998c04a1e45f37176 2023-03-08 18:54:47,628 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/db9fd55e91144c0fa343e895bcc33b96 2023-03-08 18:54:47,633 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/db9fd55e91144c0fa343e895bcc33b96 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/db9fd55e91144c0fa343e895bcc33b96 2023-03-08 18:54:47,637 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/db9fd55e91144c0fa343e895bcc33b96, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 18:54:47,638 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 828ms, sequenceid=21, compaction requested=true 2023-03-08 18:54:47,638 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:47,638 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:47,638 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:47,638 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:47,638 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,638 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:47,638 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:47,642 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,642 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:47,642 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:47,643 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:47,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:47,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@677e9df3[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:47,644 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:47,644 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,646 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,646 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:47,646 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:47,646 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:47,646 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,646 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:47,647 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:47,647 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,647 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:47,647 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:47,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:47,649 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:47,649 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:47,649 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:47,649 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,649 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:47,651 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,651 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,651 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:47,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:47,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:47,651 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,653 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:47,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:47,655 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:47,755 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:47,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:47,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:47,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:47,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,762 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:47,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:47,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:47,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:47,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,765 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:47,765 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:47,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:47,765 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:47,765 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:47,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:47,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:47,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:47,766 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:47,766 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:47,766 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:47,766 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:47,766 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:47,766 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:47,766 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:47,766 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:48,180 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512315534a2eafca840e8868c74993461f55d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512315534a2eafca840e8868c74993461f55d 2023-03-08 18:54:48,186 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/360a33cc40b24499a7bf42f745f51173 2023-03-08 18:54:48,191 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/360a33cc40b24499a7bf42f745f51173 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/360a33cc40b24499a7bf42f745f51173 2023-03-08 18:54:48,195 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/360a33cc40b24499a7bf42f745f51173, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 18:54:48,195 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 429ms, sequenceid=25, compaction requested=true 2023-03-08 18:54:48,196 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:48,196 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:48,196 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:48,196 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:48,196 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,196 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:48,196 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:48,197 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,197 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:48,198 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,198 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:48,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:48,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1014ae19[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:48,199 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:48,199 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,201 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,201 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:48,201 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:48,201 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:48,201 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,201 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:48,202 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,203 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:48,203 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:48,203 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:48,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:48,205 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:48,205 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:48,205 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:48,205 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,205 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:48,207 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,207 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:48,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,207 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:48,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:48,208 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,209 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:48,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:48,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:48,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:48,211 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:48,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:48,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,212 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:48,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:48,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:48,212 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,212 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:48,212 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,212 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,299 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:54:48,312 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:48,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:48,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:48,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:48,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,319 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:48,319 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:48,319 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:48,319 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:48,319 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,319 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,321 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:48,321 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:48,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:48,321 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:48,321 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:48,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:48,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,322 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,322 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:48,322 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:48,322 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:48,322 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:48,322 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:48,323 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:48,323 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:48,323 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:48,323 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:48,323 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:48,348 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:54:48,348 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7185c703 to 127.0.0.1:54098 2023-03-08 18:54:48,348 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:54:48,349 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:54:48,349 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06f092e5 to 127.0.0.1:54098 2023-03-08 18:54:48,349 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:54:48,349 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:54:48,349 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0831b315 to 127.0.0.1:54098 2023-03-08 18:54:48,349 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:54:48,761 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101eee798aaea764bf0a4fa3189c5eb339f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101eee798aaea764bf0a4fa3189c5eb339f 2023-03-08 18:54:49,167 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/1aa9e788ad1f4445a347a1a61b36b569 2023-03-08 18:54:49,172 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/1aa9e788ad1f4445a347a1a61b36b569 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/1aa9e788ad1f4445a347a1a61b36b569 2023-03-08 18:54:49,176 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/1aa9e788ad1f4445a347a1a61b36b569, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 18:54:49,176 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 853ms, sequenceid=29, compaction requested=true 2023-03-08 18:54:49,177 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:49,177 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:49,177 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:49,177 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:49,177 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,177 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:49,177 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:49,178 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,178 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:49,179 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,179 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:49,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:49,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@99c900f[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:49,180 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:49,180 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,181 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,181 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:49,181 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,181 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:49,181 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:49,181 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:49,184 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:49,184 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:49,184 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,184 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:49,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:49,186 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:49,186 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:49,186 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:49,186 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,186 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:49,190 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,190 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:49,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:49,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,190 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:49,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:49,191 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:49,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:49,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:49,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,192 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:49,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,195 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,195 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:49,195 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:49,195 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:49,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 18:54:49,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:49,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:49,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:49,195 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,195 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:49,196 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,196 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,196 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,196 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:49,196 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:49,196 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:49,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:49,296 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:49,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:49,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:49,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:49,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,302 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:49,302 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:49,302 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:49,302 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:49,303 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,303 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,305 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:49,305 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:49,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:49,306 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:49,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,306 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:49,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:49,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,306 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,307 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:49,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:49,307 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:49,307 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:49,307 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:49,307 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:49,307 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:49,307 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:49,307 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:49,307 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:49,720 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030b580b32c9fb4f34a1c7aee426c36405 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601030b580b32c9fb4f34a1c7aee426c36405 2023-03-08 18:54:50,126 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/7459476d9df24d82a4e798c304546e1f 2023-03-08 18:54:50,131 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/7459476d9df24d82a4e798c304546e1f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/7459476d9df24d82a4e798c304546e1f 2023-03-08 18:54:50,134 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/7459476d9df24d82a4e798c304546e1f, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 18:54:50,135 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 828ms, sequenceid=33, compaction requested=true 2023-03-08 18:54:50,135 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:50,135 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:50,135 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:50,135 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:50,135 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,135 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:50,135 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:50,137 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,137 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:50,137 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,137 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:50,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:50,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:50,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:50,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5734bf2d[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:50,139 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:50,139 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,140 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,140 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:50,140 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:50,140 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:50,140 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,140 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:50,142 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,142 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,142 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:50,142 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:50,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:50,144 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:50,144 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:50,144 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:50,144 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,144 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:50,146 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,146 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:50,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,147 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:50,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:50,147 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,148 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:50,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:50,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:50,150 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:50,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:50,150 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:50,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 18:54:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:50,151 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:50,151 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:50,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:50,251 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:50,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:50,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:50,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:50,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,257 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:50,257 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:50,258 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:50,258 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:50,258 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,258 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,259 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,259 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:50,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:50,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:50,259 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:50,259 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:50,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:50,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:50,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:50,260 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:50,260 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:50,260 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:50,261 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:50,261 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:50,261 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:50,261 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:50,261 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:50,674 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078157ded257dc4c5787d3763ba7e77b2b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509078157ded257dc4c5787d3763ba7e77b2b 2023-03-08 18:54:51,080 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/f09c062628334cf69e11a3b5f877d8dc 2023-03-08 18:54:51,084 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/f09c062628334cf69e11a3b5f877d8dc as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/f09c062628334cf69e11a3b5f877d8dc 2023-03-08 18:54:51,088 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/f09c062628334cf69e11a3b5f877d8dc, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 18:54:51,089 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 828ms, sequenceid=37, compaction requested=true 2023-03-08 18:54:51,089 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:51,089 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:51,089 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:51,089 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:51,089 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,089 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:51,089 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:51,091 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,091 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:51,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:51,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:51,091 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,091 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:51,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:51,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:51,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:51,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4436213d[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:51,093 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:51,093 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,094 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,094 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:51,094 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:51,094 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:51,094 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,094 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:51,095 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,095 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:51,095 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:51,095 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:51,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:51,098 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:51,098 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:51,098 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:51,098 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,098 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:51,100 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,100 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:51,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,100 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:51,100 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,101 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:51,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,106 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,106 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,106 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,106 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,106 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,106 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:51,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:51,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:51,106 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,106 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:51,107 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:51,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:51,107 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:51,107 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:51,107 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:51,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:51,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:51,207 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:51,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:51,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:51,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 18:54:51,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,214 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:51,214 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:51,214 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 18:54:51,214 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 18:54:51,215 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,215 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,216 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,216 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:51,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:51,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:51,216 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:51,216 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:51,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,217 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 18:54:51,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:51,217 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 18:54:51,217 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:51,217 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 18:54:51,218 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:51,218 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:51,218 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:51,218 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. started... 2023-03-08 18:54:51,218 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75fe5187a546ce484df6e7222327ef89 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:54:51,630 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511204827f107545347d78935bf14aec0b564 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204827f107545347d78935bf14aec0b564 2023-03-08 18:54:52,035 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/b127934c3fac40509f02565d17b0af35 2023-03-08 18:54:52,040 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/b127934c3fac40509f02565d17b0af35 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/b127934c3fac40509f02565d17b0af35 2023-03-08 18:54:52,044 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/b127934c3fac40509f02565d17b0af35, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 18:54:52,045 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 75fe5187a546ce484df6e7222327ef89 in 827ms, sequenceid=41, compaction requested=true 2023-03-08 18:54:52,045 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:52,045 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:52,046 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:54:52,046 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:54:52,046 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,046 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 18:54:52,046 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 18:54:52,048 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,048 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:52,048 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:52,048 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 18:54:52,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71e73bc4[Count = 0] remaining members to acquire global barrier 2023-03-08 18:54:52,050 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 18:54:52,050 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,051 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,051 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 18:54:52,051 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 18:54:52,051 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:54:52,051 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,051 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:54:52,052 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 18:54:52,052 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:52,052 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,052 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 18:54:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 18:54:52,054 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 18:54:52,055 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:54:52,055 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:54:52,055 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,055 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:54:52,056 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,056 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,056 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:54:52,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:54:52,056 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,057 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,060 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,060 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,060 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,060 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:52,060 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 18:54:52,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:54:52,061 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:52,061 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:52,061 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:54:52,061 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 18:54:52,061 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,061 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:54:52,061 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:54:52,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:54:52,161 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:54:52,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:54:52,164 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:52,165 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:52,166 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:52,167 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302292167 2023-03-08 18:54:52,167 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:52,168 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:54:52,168 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 18:54:52,168 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:52,169 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x67a79a92 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:54:52,173 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7bea9bd3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:54:52,174 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,024 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103afa9a9205144462abc09baf000ddcdf5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103afa9a9205144462abc09baf000ddcdf5 2023-03-08 18:54:53,025 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:53,042 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:54:53,043 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:58732, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:54:53,050 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:54:53,052 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:39154, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:54:53,062 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0046c4ea0671441aaa8776c792b3de9c first=Optional[r3] last=Optional[r6] 2023-03-08 18:54:53,063 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0046c4ea0671441aaa8776c792b3de9c}] 2023-03-08 18:54:53,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0046c4ea0671441aaa8776c792b3de9c for inclusion in 75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:53,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 18:54:53,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:53,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:53,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0046c4ea0671441aaa8776c792b3de9c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hv7l1ueol2fssbiq01ds5r2c1uq6a69o1vhb12njsrf58c7gvmcvch86e47oriaq/family1/0046c4ea0671441aaa8776c792b3de9c 2023-03-08 18:54:53,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hv7l1ueol2fssbiq01ds5r2c1uq6a69o1vhb12njsrf58c7gvmcvch86e47oriaq/family1/0046c4ea0671441aaa8776c792b3de9c as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/27164d9bbac64dfb82f5c354d36b2029_SeqId_44_ 2023-03-08 18:54:53,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hv7l1ueol2fssbiq01ds5r2c1uq6a69o1vhb12njsrf58c7gvmcvch86e47oriaq/family1/0046c4ea0671441aaa8776c792b3de9c into 75fe5187a546ce484df6e7222327ef89/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/27164d9bbac64dfb82f5c354d36b2029_SeqId_44_ - updating store file list. 2023-03-08 18:54:53,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/27164d9bbac64dfb82f5c354d36b2029_SeqId_44_ into 75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:53,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hv7l1ueol2fssbiq01ds5r2c1uq6a69o1vhb12njsrf58c7gvmcvch86e47oriaq/family1/0046c4ea0671441aaa8776c792b3de9c into 75fe5187a546ce484df6e7222327ef89/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/27164d9bbac64dfb82f5c354d36b2029_SeqId_44_) 2023-03-08 18:54:53,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hv7l1ueol2fssbiq01ds5r2c1uq6a69o1vhb12njsrf58c7gvmcvch86e47oriaq/family1/0046c4ea0671441aaa8776c792b3de9c 2023-03-08 18:54:53,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 18:54:53,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 18:54:53,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): 75fe5187a546ce484df6e7222327ef89/family1 is initiating minor compaction 2023-03-08 18:54:53,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@13b7e8be; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:53,084 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 75fe5187a546ce484df6e7222327ef89/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:53,085 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/baa5198c024c4ac78178d4e66adf06a5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/c1622c9ef63049c69045098049b51e64, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2d9e7471cb0141a29353e3b1ba3ffb4c, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/db9fd55e91144c0fa343e895bcc33b96, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/360a33cc40b24499a7bf42f745f51173, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/1aa9e788ad1f4445a347a1a61b36b569, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/7459476d9df24d82a4e798c304546e1f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/f09c062628334cf69e11a3b5f877d8dc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/b127934c3fac40509f02565d17b0af35, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/27164d9bbac64dfb82f5c354d36b2029_SeqId_44_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp, totalSize=50.4 K 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting baa5198c024c4ac78178d4e66adf06a5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting c1622c9ef63049c69045098049b51e64, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 2d9e7471cb0141a29353e3b1ba3ffb4c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting db9fd55e91144c0fa343e895bcc33b96, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 360a33cc40b24499a7bf42f745f51173, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 1aa9e788ad1f4445a347a1a61b36b569, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 7459476d9df24d82a4e798c304546e1f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting f09c062628334cf69e11a3b5f877d8dc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting b127934c3fac40509f02565d17b0af35, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 18:54:53,085 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 27164d9bbac64dfb82f5c354d36b2029_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 18:54:53,086 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:53,087 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:53,089 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122839c3a8fd54b44fc998c04a1e45f37176 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122839c3a8fd54b44fc998c04a1e45f37176 2023-03-08 18:54:53,090 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512315534a2eafca840e8868c74993461f55d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512315534a2eafca840e8868c74993461f55d 2023-03-08 18:54:53,091 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101eee798aaea764bf0a4fa3189c5eb339f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101eee798aaea764bf0a4fa3189c5eb339f 2023-03-08 18:54:53,092 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601030b580b32c9fb4f34a1c7aee426c36405 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601030b580b32c9fb4f34a1c7aee426c36405 2023-03-08 18:54:53,092 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-08 18:54:53,092 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 18:54:53,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,115 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 75fe5187a546ce484df6e7222327ef89#family1#compaction#61 average throughput is 0.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:53,543 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/5fe3258089cf44f8baeac1aec38546c6 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/5fe3258089cf44f8baeac1aec38546c6 2023-03-08 18:54:53,548 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 75fe5187a546ce484df6e7222327ef89/family1 of 75fe5187a546ce484df6e7222327ef89 into 5fe3258089cf44f8baeac1aec38546c6(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:54:53,548 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:53,549 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678301693084; duration=0sec 2023-03-08 18:54:53,549 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:53,591 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/baa5198c024c4ac78178d4e66adf06a5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/c1622c9ef63049c69045098049b51e64, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2d9e7471cb0141a29353e3b1ba3ffb4c, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/db9fd55e91144c0fa343e895bcc33b96, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/360a33cc40b24499a7bf42f745f51173, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/1aa9e788ad1f4445a347a1a61b36b569, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/7459476d9df24d82a4e798c304546e1f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/f09c062628334cf69e11a3b5f877d8dc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/b127934c3fac40509f02565d17b0af35, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/27164d9bbac64dfb82f5c354d36b2029_SeqId_44_] to archive 2023-03-08 18:54:53,592 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:53,594 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/baa5198c024c4ac78178d4e66adf06a5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/baa5198c024c4ac78178d4e66adf06a5 2023-03-08 18:54:53,596 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/c1622c9ef63049c69045098049b51e64 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/c1622c9ef63049c69045098049b51e64 2023-03-08 18:54:53,597 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2d9e7471cb0141a29353e3b1ba3ffb4c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2d9e7471cb0141a29353e3b1ba3ffb4c 2023-03-08 18:54:53,598 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/db9fd55e91144c0fa343e895bcc33b96 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/db9fd55e91144c0fa343e895bcc33b96 2023-03-08 18:54:53,600 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/360a33cc40b24499a7bf42f745f51173 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/360a33cc40b24499a7bf42f745f51173 2023-03-08 18:54:53,601 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/1aa9e788ad1f4445a347a1a61b36b569 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/1aa9e788ad1f4445a347a1a61b36b569 2023-03-08 18:54:53,602 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/7459476d9df24d82a4e798c304546e1f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/7459476d9df24d82a4e798c304546e1f 2023-03-08 18:54:53,603 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/f09c062628334cf69e11a3b5f877d8dc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/f09c062628334cf69e11a3b5f877d8dc 2023-03-08 18:54:53,604 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/b127934c3fac40509f02565d17b0af35 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/b127934c3fac40509f02565d17b0af35 2023-03-08 18:54:53,607 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/27164d9bbac64dfb82f5c354d36b2029_SeqId_44_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/27164d9bbac64dfb82f5c354d36b2029_SeqId_44_ 2023-03-08 18:54:53,950 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205adddaf8468b646788961e25fee332b58 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205adddaf8468b646788961e25fee332b58 2023-03-08 18:54:53,951 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:53,974 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bd13ecb66d394cbc940f1bc8b5e07374 first=Optional[r00] last=Optional[r2] 2023-03-08 18:54:53,975 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bd13ecb66d394cbc940f1bc8b5e07374}] 2023-03-08 18:54:53,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bd13ecb66d394cbc940f1bc8b5e07374 for inclusion in 75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:53,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 18:54:53,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:53,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:53,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bd13ecb66d394cbc940f1bc8b5e07374 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p1vi05sekpsf0av5q3s4e18fsu75mjqm4kck5m0o6hoppq3gpi2gpe6vt7efi12r/family1/bd13ecb66d394cbc940f1bc8b5e07374 2023-03-08 18:54:53,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p1vi05sekpsf0av5q3s4e18fsu75mjqm4kck5m0o6hoppq3gpi2gpe6vt7efi12r/family1/bd13ecb66d394cbc940f1bc8b5e07374 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_ 2023-03-08 18:54:53,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p1vi05sekpsf0av5q3s4e18fsu75mjqm4kck5m0o6hoppq3gpi2gpe6vt7efi12r/family1/bd13ecb66d394cbc940f1bc8b5e07374 into 75fe5187a546ce484df6e7222327ef89/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_ - updating store file list. 2023-03-08 18:54:53,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_ into 75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:53,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p1vi05sekpsf0av5q3s4e18fsu75mjqm4kck5m0o6hoppq3gpi2gpe6vt7efi12r/family1/bd13ecb66d394cbc940f1bc8b5e07374 into 75fe5187a546ce484df6e7222327ef89/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_) 2023-03-08 18:54:53,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p1vi05sekpsf0av5q3s4e18fsu75mjqm4kck5m0o6hoppq3gpi2gpe6vt7efi12r/family1/bd13ecb66d394cbc940f1bc8b5e07374 2023-03-08 18:54:53,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 18:54:53,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 18:54:53,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): 75fe5187a546ce484df6e7222327ef89/family1 is initiating minor compaction (all files) 2023-03-08 18:54:53,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3b0e3e40; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:53,989 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 75fe5187a546ce484df6e7222327ef89/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:53,989 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0738496e548e4e578ef18f1a3d685870, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/5fe3258089cf44f8baeac1aec38546c6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp, totalSize=390.1 K 2023-03-08 18:54:53,990 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 0738496e548e4e578ef18f1a3d685870, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845940 2023-03-08 18:54:53,990 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 5fe3258089cf44f8baeac1aec38546c6, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245940 2023-03-08 18:54:53,991 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845940 2023-03-08 18:54:53,991 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:53,992 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:53,993 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130ec29e013383d48669fba4b4b86f8756e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130ec29e013383d48669fba4b4b86f8756e 2023-03-08 18:54:53,994 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201beba1d3b171c4b789837cae82bf86180 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201beba1d3b171c4b789837cae82bf86180 2023-03-08 18:54:53,996 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d50a8e14704e4037bd5703bc81671e33 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d50a8e14704e4037bd5703bc81671e33 2023-03-08 18:54:53,996 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-08 18:54:53,996 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 18:54:53,996 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:53,996 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:53,996 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:53,997 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:53,999 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8330 sec 2023-03-08 18:54:54,009 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 75fe5187a546ce484df6e7222327ef89#family1#compaction#62 average throughput is 20.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:54:54,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$12(2454): Client=jenkins//172.31.2.20 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 18:54:54,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:54,482 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 18:54:54,483 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-08 18:54:54,483 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 18:54:54,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-08 18:54:54,487 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, REOPEN/MOVE}] 2023-03-08 18:54:54,488 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, REOPEN/MOVE 2023-03-08 18:54:54,489 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:54,489 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301694489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301694489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301694489"}]},"ts":"1678301694489"} 2023-03-08 18:54:54,490 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:54,642 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:54,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 75fe5187a546ce484df6e7222327ef89, disabling compactions & flushes 2023-03-08 18:54:54,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:55,227 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/.tmp/family1/2b87e48072ff41adb9b97db147db4589 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2b87e48072ff41adb9b97db147db4589 2023-03-08 18:54:55,232 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 75fe5187a546ce484df6e7222327ef89/family1 of 75fe5187a546ce484df6e7222327ef89 into 2b87e48072ff41adb9b97db147db4589(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:54:55,232 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:55,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:55,232 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1678301693989; duration=1sec 2023-03-08 18:54:55,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:55,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. after waiting 0 ms 2023-03-08 18:54:55,232 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:54:55,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:55,234 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0738496e548e4e578ef18f1a3d685870, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/5fe3258089cf44f8baeac1aec38546c6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_] to archive 2023-03-08 18:54:55,235 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:55,236 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0738496e548e4e578ef18f1a3d685870 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0738496e548e4e578ef18f1a3d685870 2023-03-08 18:54:55,237 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/5fe3258089cf44f8baeac1aec38546c6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/5fe3258089cf44f8baeac1aec38546c6 2023-03-08 18:54:55,238 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/ff071a3a9a974049ba2aab11eea7d8aa_SeqId_47_ 2023-03-08 18:54:55,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 18:54:55,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:55,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:55,243 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 75fe5187a546ce484df6e7222327ef89 to self. 2023-03-08 18:54:55,245 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:55,245 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=CLOSED 2023-03-08 18:54:55,245 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301695245"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301695245"}]},"ts":"1678301695245"} 2023-03-08 18:54:55,248 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-08 18:54:55,248 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627 in 756 msec 2023-03-08 18:54:55,248 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=true 2023-03-08 18:54:55,399 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:55,399 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301695399"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301695399"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301695399"}]},"ts":"1678301695399"} 2023-03-08 18:54:55,401 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:55,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:55,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75fe5187a546ce484df6e7222327ef89, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:54:55,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:55,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:55,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:55,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:55,557 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:55,558 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:55,558 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:55,558 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75fe5187a546ce484df6e7222327ef89 columnFamilyName family1 2023-03-08 18:54:55,563 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2b87e48072ff41adb9b97db147db4589 2023-03-08 18:54:55,563 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(310): Store=75fe5187a546ce484df6e7222327ef89/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:55,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:55,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:55,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:55,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 75fe5187a546ce484df6e7222327ef89; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68465830, jitterRate=0.02022036910057068}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:54:55,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:55,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89., pid=47, masterSystemTime=1678301695552 2023-03-08 18:54:55,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:55,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:55,573 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:55,573 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301695573"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301695573"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301695573"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301695573"}]},"ts":"1678301695573"} 2023-03-08 18:54:55,576 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-08 18:54:55,576 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627 in 174 msec 2023-03-08 18:54:55,577 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-08 18:54:55,577 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, REOPEN/MOVE in 1.0890 sec 2023-03-08 18:54:55,579 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-08 18:54:55,579 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0930 sec 2023-03-08 18:54:55,581 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5080 sec 2023-03-08 18:54:55,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-08 18:54:55,583 INFO [Listener at localhost/36281] compactions.TestMobCompactor(278): alter status finished 2023-03-08 18:54:55,584 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:55,585 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:55,586 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:55,586 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:55,586 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302295586 2023-03-08 18:54:55,587 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:54:55,588 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 18:54:55,588 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:55,588 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x494d79f7 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:54:55,593 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6656470e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:54:55,593 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:55,593 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:55,593 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:55,593 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:55,593 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:56,424 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a557d4fe9d574282abde41db0942219d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128a557d4fe9d574282abde41db0942219d 2023-03-08 18:54:56,425 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:56,443 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:54:56,444 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:58742, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:54:56,452 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:54:56,454 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:39168, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:54:56,463 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/35fc87c97fe445d287f08cc7193309e5 first=Optional[r05] last=Optional[r8] 2023-03-08 18:54:56,463 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=51 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/35fc87c97fe445d287f08cc7193309e5}] 2023-03-08 18:54:56,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/35fc87c97fe445d287f08cc7193309e5 for inclusion in 75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:56,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 18:54:56,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:54:56,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:56,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/35fc87c97fe445d287f08cc7193309e5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ra6h7saq9ei40gebovcibdt9dsod6h3r4ivr670m2fadn7o2e9rnmh4l9lb3o4e3/family1/35fc87c97fe445d287f08cc7193309e5 2023-03-08 18:54:56,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ra6h7saq9ei40gebovcibdt9dsod6h3r4ivr670m2fadn7o2e9rnmh4l9lb3o4e3/family1/35fc87c97fe445d287f08cc7193309e5 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0d46b56694b144478881d9cc29a9a7cb_SeqId_53_ 2023-03-08 18:54:56,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ra6h7saq9ei40gebovcibdt9dsod6h3r4ivr670m2fadn7o2e9rnmh4l9lb3o4e3/family1/35fc87c97fe445d287f08cc7193309e5 into 75fe5187a546ce484df6e7222327ef89/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0d46b56694b144478881d9cc29a9a7cb_SeqId_53_ - updating store file list. 2023-03-08 18:54:56,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0d46b56694b144478881d9cc29a9a7cb_SeqId_53_ into 75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:56,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ra6h7saq9ei40gebovcibdt9dsod6h3r4ivr670m2fadn7o2e9rnmh4l9lb3o4e3/family1/35fc87c97fe445d287f08cc7193309e5 into 75fe5187a546ce484df6e7222327ef89/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0d46b56694b144478881d9cc29a9a7cb_SeqId_53_) 2023-03-08 18:54:56,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ra6h7saq9ei40gebovcibdt9dsod6h3r4ivr670m2fadn7o2e9rnmh4l9lb3o4e3/family1/35fc87c97fe445d287f08cc7193309e5 2023-03-08 18:54:56,480 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:56,481 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:54:56,482 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204827f107545347d78935bf14aec0b564 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204827f107545347d78935bf14aec0b564 2023-03-08 18:54:56,484 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128ab7e7d196b1a43cdbc8b04a1aef0f421 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128ab7e7d196b1a43cdbc8b04a1aef0f421 2023-03-08 18:54:56,484 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-08 18:54:56,484 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 18:54:56,484 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:56,484 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:56,484 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:56,484 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:56,484 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:56,486 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:56,487 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 902 msec 2023-03-08 18:54:56,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$12(2454): Client=jenkins//172.31.2.20 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 18:54:56,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:56,951 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-08 18:54:56,951 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 18:54:56,952 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-08 18:54:56,953 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-08 18:54:56,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, REOPEN/MOVE}] 2023-03-08 18:54:56,957 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, REOPEN/MOVE 2023-03-08 18:54:56,958 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:56,958 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301696958"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301696958"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301696958"}]},"ts":"1678301696958"} 2023-03-08 18:54:56,959 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:57,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 75fe5187a546ce484df6e7222327ef89, disabling compactions & flushes 2023-03-08 18:54:57,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:57,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:57,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. after waiting 0 ms 2023-03-08 18:54:57,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:57,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 18:54:57,118 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:57,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:57,118 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 75fe5187a546ce484df6e7222327ef89 to self. 2023-03-08 18:54:57,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,119 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=CLOSED 2023-03-08 18:54:57,120 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301697119"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301697119"}]},"ts":"1678301697119"} 2023-03-08 18:54:57,122 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-08 18:54:57,122 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627 in 162 msec 2023-03-08 18:54:57,123 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=true 2023-03-08 18:54:57,273 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:57,273 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301697273"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301697273"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301697273"}]},"ts":"1678301697273"} 2023-03-08 18:54:57,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:57,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:57,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75fe5187a546ce484df6e7222327ef89, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:54:57,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:57,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,431 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,431 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:57,431 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1 2023-03-08 18:54:57,432 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75fe5187a546ce484df6e7222327ef89 columnFamilyName family1 2023-03-08 18:54:57,436 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/0d46b56694b144478881d9cc29a9a7cb_SeqId_53_ 2023-03-08 18:54:57,439 DEBUG [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/family1/2b87e48072ff41adb9b97db147db4589 2023-03-08 18:54:57,439 INFO [StoreOpener-75fe5187a546ce484df6e7222327ef89-1] regionserver.HStore(310): Store=75fe5187a546ce484df6e7222327ef89/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:57,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:54:57,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 75fe5187a546ce484df6e7222327ef89; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62268324, jitterRate=-0.07212966680526733}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:54:57,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:54:57,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89., pid=53, masterSystemTime=1678301697426 2023-03-08 18:54:57,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:57,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:54:57,446 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=75fe5187a546ce484df6e7222327ef89, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:57,446 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678301697446"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301697446"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301697446"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301697446"}]},"ts":"1678301697446"} 2023-03-08 18:54:57,448 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-08 18:54:57,448 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 75fe5187a546ce484df6e7222327ef89, server=jenkins-hbase6.apache.org,46397,1678301621627 in 172 msec 2023-03-08 18:54:57,450 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-08 18:54:57,450 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=75fe5187a546ce484df6e7222327ef89, REOPEN/MOVE in 492 msec 2023-03-08 18:54:57,451 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-08 18:54:57,451 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 497 msec 2023-03-08 18:54:57,453 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 919 msec 2023-03-08 18:54:57,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-08 18:54:57,455 INFO [Listener at localhost/36281] compactions.TestMobCompactor(278): alter status finished 2023-03-08 18:54:57,456 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 18:54:57,457 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:57,457 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:57,458 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:57,458 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302297458 2023-03-08 18:54:57,458 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:54:57,459 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 18:54:57,459 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:57,459 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x40a99ac2 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:54:57,463 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1413cbee, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:54:57,463 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:57,463 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:57,463 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:57,463 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:57,463 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:57,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:57,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:57,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 18:54:57,464 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 18:54:57,464 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:54:57,464 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 18:54:57,465 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 8 msec 2023-03-08 18:54:57,517 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=242 (was 253), OpenFileDescriptor=611 (was 617), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=170 (was 198), ProcessCount=172 (was 178), AvailableMemoryMB=7057 (was 7120) 2023-03-08 18:54:57,528 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=242, OpenFileDescriptor=611, MaxFileDescriptor=60000, SystemLoadAverage=170, ProcessCount=172, AvailableMemoryMB=7057 2023-03-08 18:54:57,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 18:54:57,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:57,532 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:54:57,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-08 18:54:57,532 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:54:57,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 18:54:57,533 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:57,534 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 empty. 2023-03-08 18:54:57,534 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:57,534 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-08 18:54:57,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 18:54:57,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 18:54:57,942 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-08 18:54:57,943 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => fb8791aea8f6b5c60c5c17523aa003f3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:54:58,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 18:54:58,349 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:58,349 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing fb8791aea8f6b5c60c5c17523aa003f3, disabling compactions & flushes 2023-03-08 18:54:58,349 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:54:58,349 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:54:58,350 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. after waiting 0 ms 2023-03-08 18:54:58,350 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:54:58,350 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:54:58,350 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:54:58,352 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:54:58,352 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301698352"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301698352"}]},"ts":"1678301698352"} 2023-03-08 18:54:58,354 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:54:58,354 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:54:58,354 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301698354"}]},"ts":"1678301698354"} 2023-03-08 18:54:58,355 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-08 18:54:58,360 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, ASSIGN}] 2023-03-08 18:54:58,361 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, ASSIGN 2023-03-08 18:54:58,362 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:54:58,513 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:58,513 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301698513"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301698513"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301698513"}]},"ts":"1678301698513"} 2023-03-08 18:54:58,515 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:54:58,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 18:54:58,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:54:58,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fb8791aea8f6b5c60c5c17523aa003f3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:54:58,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:58,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:54:58,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:58,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:58,671 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:58,672 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:54:58,672 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:54:58,672 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb8791aea8f6b5c60c5c17523aa003f3 columnFamilyName family1 2023-03-08 18:54:58,674 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(310): Store=fb8791aea8f6b5c60c5c17523aa003f3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:54:58,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:58,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:58,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:54:58,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:54:58,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened fb8791aea8f6b5c60c5c17523aa003f3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62258540, jitterRate=-0.07227545976638794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:54:58,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:54:58,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3., pid=57, masterSystemTime=1678301698666 2023-03-08 18:54:58,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:54:58,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:54:58,682 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:58,682 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301698682"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301698682"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301698682"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301698682"}]},"ts":"1678301698682"} 2023-03-08 18:54:58,684 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-08 18:54:58,684 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627 in 169 msec 2023-03-08 18:54:58,685 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-08 18:54:58,686 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, ASSIGN in 325 msec 2023-03-08 18:54:58,686 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:54:58,686 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301698686"}]},"ts":"1678301698686"} 2023-03-08 18:54:58,687 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-08 18:54:58,689 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:54:58,690 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1590 sec 2023-03-08 18:54:59,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 18:54:59,638 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-08 18:54:59,666 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:54:59,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:59,668 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:54:59,668 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:59,668 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:54:59,668 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:54:59,669 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:59,669 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:59,670 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:54:59,670 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:59,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:54:59,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:54:59,670 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:54:59,670 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:54:59,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:59,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:59,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:54:59,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:59,671 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:59,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:54:59,672 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:54:59,672 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:54:59,672 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:54:59,675 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:54:59,675 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:54:59,675 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:54:59,675 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:54:59,676 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 18:55:00,103 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130337f0f1627aa4ff9bbd3864a380da010 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130337f0f1627aa4ff9bbd3864a380da010 2023-03-08 18:55:00,506 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/38b10b4b1bce4551889e68f6c47345a3 2023-03-08 18:55:00,511 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/38b10b4b1bce4551889e68f6c47345a3 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/38b10b4b1bce4551889e68f6c47345a3 2023-03-08 18:55:00,515 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/38b10b4b1bce4551889e68f6c47345a3, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-08 18:55:00,516 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 840ms, sequenceid=5, compaction requested=false 2023-03-08 18:55:00,516 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:00,516 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:00,516 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:00,516 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:00,516 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:00,516 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,517 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:00,517 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:00,519 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,519 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:00,519 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,519 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:00,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:00,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:00,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:00,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c16c11f[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:00,521 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:00,521 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,522 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,522 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:00,522 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:00,522 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:00,522 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,522 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:00,523 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:00,523 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,523 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:00,523 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:00,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:00,525 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:00,525 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:00,525 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:00,525 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:00,525 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,525 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:00,527 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,527 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,527 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:00,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:00,528 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,528 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,539 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:00,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:00,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:00,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:00,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:00,539 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:00,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:00,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:00,640 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:00,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:00,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:00,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:00,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,646 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:00,646 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:00,646 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:00,646 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:00,647 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,647 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:00,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:00,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:00,648 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,649 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:00,649 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:00,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:00,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,650 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:00,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:00,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:00,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:00,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:00,653 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:00,653 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:00,653 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:00,653 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:00,653 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 18:55:00,772 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:55:01,068 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511286f67f2c98ef945deac81fe2440cd4c0d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511286f67f2c98ef945deac81fe2440cd4c0d 2023-03-08 18:55:01,475 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/3a819bf1c4fb4cb1b80b525ccfbc11d2 2023-03-08 18:55:01,479 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/3a819bf1c4fb4cb1b80b525ccfbc11d2 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/3a819bf1c4fb4cb1b80b525ccfbc11d2 2023-03-08 18:55:01,482 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/3a819bf1c4fb4cb1b80b525ccfbc11d2, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 18:55:01,483 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 830ms, sequenceid=9, compaction requested=false 2023-03-08 18:55:01,483 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:01,483 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:01,483 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:01,483 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:01,483 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,483 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:01,483 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:01,485 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,485 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:01,485 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:01,485 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:01,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:01,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:01,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e29c867[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:01,486 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:01,487 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,488 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,488 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:01,488 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:01,488 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:01,488 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,488 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:01,490 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:01,490 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,490 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:01,490 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:01,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:01,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:01,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:01,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:01,492 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:01,492 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:01,492 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:01,492 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,492 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:01,494 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,494 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:01,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,494 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:01,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:01,494 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:01,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,495 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:01,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:01,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:01,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:01,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:01,497 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:01,497 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:01,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:01,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,498 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:01,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,498 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:01,498 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,598 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:01,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:01,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:01,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:01,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,605 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:01,605 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:01,605 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:01,605 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:01,606 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,606 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,607 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:01,607 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:01,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:01,608 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:01,608 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:01,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,608 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:01,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,608 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,609 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:01,609 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:01,609 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:01,609 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:01,609 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:01,609 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:01,609 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:01,609 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:01,609 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:01,609 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:55:02,024 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201bf3aca91571e463d8caade8c5d5aa4fb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201bf3aca91571e463d8caade8c5d5aa4fb 2023-03-08 18:55:02,430 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/c7e8eee14f3c46e9a44fdd97d5f79cbb 2023-03-08 18:55:02,434 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/c7e8eee14f3c46e9a44fdd97d5f79cbb as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/c7e8eee14f3c46e9a44fdd97d5f79cbb 2023-03-08 18:55:02,438 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/c7e8eee14f3c46e9a44fdd97d5f79cbb, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 18:55:02,439 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 830ms, sequenceid=13, compaction requested=true 2023-03-08 18:55:02,439 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:02,439 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:02,439 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:02,439 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:02,439 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,439 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:02,439 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:02,442 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,442 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:02,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:02,442 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,442 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:02,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:02,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:02,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:02,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:02,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b633356[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:02,444 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:02,444 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,445 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,445 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:02,445 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:02,445 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:02,445 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,445 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:02,446 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,446 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:02,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:02,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:02,447 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:02,447 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:02,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:02,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:02,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:02,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:02,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:02,449 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:02,449 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:02,449 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:02,449 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,449 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:02,450 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,450 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:02,450 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:02,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:02,451 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,452 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,455 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,455 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:02,455 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:02,455 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:02,455 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:02,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 18:55:02,455 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:02,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:02,455 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,456 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,456 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,456 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,456 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:02,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:02,456 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:02,456 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:02,556 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:02,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:02,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:02,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:02,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,563 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:02,563 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:02,563 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:02,563 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:02,564 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,564 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,565 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:02,565 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:02,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:02,566 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:02,566 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,566 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:02,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:02,567 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:02,567 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:02,567 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:02,567 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:02,567 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:02,567 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:02,567 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:02,567 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:55:02,980 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a5adc98e040249c6867f16dcbbd3d14d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a5adc98e040249c6867f16dcbbd3d14d 2023-03-08 18:55:03,385 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/55a6df29d30d40b9844c3b2bcc32e34b 2023-03-08 18:55:03,389 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/55a6df29d30d40b9844c3b2bcc32e34b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/55a6df29d30d40b9844c3b2bcc32e34b 2023-03-08 18:55:03,393 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/55a6df29d30d40b9844c3b2bcc32e34b, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 18:55:03,394 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 827ms, sequenceid=17, compaction requested=true 2023-03-08 18:55:03,394 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:03,394 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:03,394 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:03,394 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:03,394 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,394 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:03,394 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:03,396 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,396 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:03,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:03,396 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,396 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:03,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:03,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:03,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:03,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:03,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@531212e5[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:03,397 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:03,398 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,402 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:03,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:03,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:03,402 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,402 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:03,404 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:03,404 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:03,404 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,404 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:03,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:03,406 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:03,406 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:03,406 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:03,406 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,406 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:03,408 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,408 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:03,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:03,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,408 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:03,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:03,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:03,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:03,408 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:03,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,409 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:03,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:03,412 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:03,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:03,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:03,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:03,412 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:03,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:03,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:03,513 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:03,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:03,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:03,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:03,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,519 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:03,519 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:03,519 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:03,519 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:03,520 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,520 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,522 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:03,522 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:03,522 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:03,522 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,523 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:03,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:03,523 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:03,523 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:03,524 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:03,524 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:03,524 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:03,524 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:03,524 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:03,524 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:55:03,939 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512288ce5915ef8c44f7db55afa49da2284f4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512288ce5915ef8c44f7db55afa49da2284f4 2023-03-08 18:55:04,344 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/d9537e83994a4bd4bdf9993173fe3439 2023-03-08 18:55:04,349 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/d9537e83994a4bd4bdf9993173fe3439 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/d9537e83994a4bd4bdf9993173fe3439 2023-03-08 18:55:04,353 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/d9537e83994a4bd4bdf9993173fe3439, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 18:55:04,354 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 830ms, sequenceid=21, compaction requested=true 2023-03-08 18:55:04,354 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:04,354 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:04,354 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:04,354 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:04,354 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,354 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:04,354 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:04,356 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,356 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:04,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:04,356 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,356 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:04,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:04,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:04,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:04,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:04,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79263fa1[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:04,357 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:04,357 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,358 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,358 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:04,359 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:04,359 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:04,359 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,359 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:04,360 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:04,360 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,360 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:04,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:04,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:04,360 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:04,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:04,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:04,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:04,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:04,362 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:04,362 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:04,362 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:04,362 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,362 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:04,364 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,364 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:04,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:04,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,364 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:04,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:04,365 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:04,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,366 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:04,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:04,368 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:04,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 18:55:04,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:04,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:04,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:04,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,368 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:04,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:04,369 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:04,369 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:04,469 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:04,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:04,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:04,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:04,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,475 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:04,475 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:04,475 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:04,475 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:04,475 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,475 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,477 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:04,477 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:04,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:04,478 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:04,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,478 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:04,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:04,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,478 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:04,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:04,479 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:04,479 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:04,479 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:04,479 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:04,479 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:04,479 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:04,479 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:04,479 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:55:04,893 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123138de077309954e0aa457cb365a9c5ee9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123138de077309954e0aa457cb365a9c5ee9 2023-03-08 18:55:05,300 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/2d106608bb8f4174afca4a883bcf8493 2023-03-08 18:55:05,305 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/2d106608bb8f4174afca4a883bcf8493 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/2d106608bb8f4174afca4a883bcf8493 2023-03-08 18:55:05,308 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/2d106608bb8f4174afca4a883bcf8493, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 18:55:05,309 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 830ms, sequenceid=25, compaction requested=true 2023-03-08 18:55:05,309 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:05,309 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:05,309 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:05,309 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:05,309 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,309 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:05,310 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:05,311 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,311 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:05,311 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,311 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:05,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:05,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@596caa12[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:05,313 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:05,313 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,315 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,316 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:05,316 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:05,316 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:05,316 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,316 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:05,317 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:05,317 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,317 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:05,317 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:05,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:05,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:05,319 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:05,319 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:05,319 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:05,319 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,319 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:05,321 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,321 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:05,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:05,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,321 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:05,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:05,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:05,322 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:05,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:05,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,323 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:05,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,328 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,328 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,328 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:05,328 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,329 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:05,329 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:05,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 18:55:05,329 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:05,329 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:05,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:05,329 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:05,329 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,329 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:05,329 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:05,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:05,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:05,429 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:05,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:05,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:05,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:05,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,435 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:05,435 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:05,435 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:05,435 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:05,435 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,435 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,442 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:05,442 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:05,443 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:05,443 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,443 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,444 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:05,444 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:05,444 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:05,444 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:05,444 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:05,444 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:05,444 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:05,444 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:05,444 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:55:05,856 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018745ee8c87464289814431a21763e9df to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018745ee8c87464289814431a21763e9df 2023-03-08 18:55:06,261 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/f2b0e4800ce8445bab9b842d43fd7ed9 2023-03-08 18:55:06,266 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/f2b0e4800ce8445bab9b842d43fd7ed9 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f2b0e4800ce8445bab9b842d43fd7ed9 2023-03-08 18:55:06,271 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f2b0e4800ce8445bab9b842d43fd7ed9, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 18:55:06,272 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 828ms, sequenceid=29, compaction requested=true 2023-03-08 18:55:06,272 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:06,272 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:06,272 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:06,272 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:06,272 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,272 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:06,272 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:06,274 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,274 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:06,274 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,274 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:06,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:06,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b97dbea[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:06,275 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:06,275 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,276 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,277 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:06,277 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:06,277 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:06,277 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,277 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:06,279 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,279 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:06,279 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:06,279 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:06,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:06,281 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:06,281 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:06,281 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:06,281 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:06,281 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,281 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:06,282 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,282 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:06,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:06,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,282 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:06,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:06,283 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:06,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:06,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:06,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,284 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:06,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,286 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:06,286 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:06,286 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,286 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:06,286 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,287 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:06,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:06,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:06,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:06,287 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,287 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,287 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,287 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:06,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:06,287 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:06,287 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:06,387 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:06,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:06,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:06,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:06,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,394 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:06,394 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:06,394 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:06,394 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:06,395 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,395 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,396 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:06,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:06,396 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,397 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,397 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:06,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,402 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:06,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:06,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:06,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:06,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:06,403 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:06,403 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:06,403 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:06,403 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:55:06,820 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103dbae00097e5f4e9b8946af8438446506 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103dbae00097e5f4e9b8946af8438446506 2023-03-08 18:55:06,826 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/335d7ce1024c44408d82e0cce687db8b 2023-03-08 18:55:06,829 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/335d7ce1024c44408d82e0cce687db8b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/335d7ce1024c44408d82e0cce687db8b 2023-03-08 18:55:06,833 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/335d7ce1024c44408d82e0cce687db8b, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 18:55:06,833 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 430ms, sequenceid=33, compaction requested=true 2023-03-08 18:55:06,833 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:06,834 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:06,834 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:06,834 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:06,834 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,834 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:06,834 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:06,836 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,836 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:06,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:06,836 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:06,836 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:06,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:06,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:06,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:06,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18d16f47[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:06,837 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:06,837 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,838 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,839 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:06,839 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:06,839 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:06,839 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,839 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:06,840 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,840 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:06,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:06,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:06,840 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:06,840 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:06,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:06,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:06,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:06,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:06,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:06,842 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:06,842 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:06,842 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:06,842 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,842 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:06,844 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,844 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:06,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:06,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,844 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:06,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:06,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:06,844 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:06,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:06,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,845 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:06,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:06,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,848 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:06,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 18:55:06,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:06,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:06,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:06,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:06,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:06,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,849 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,849 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,849 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,849 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,849 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:06,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:06,849 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:06,849 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:06,949 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:06,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:06,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:06,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:06,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,956 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:06,956 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:06,956 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:06,956 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:06,957 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,957 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,958 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,958 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:06,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:06,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:06,959 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:06,959 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:06,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:06,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,960 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,960 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:06,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:06,960 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:06,960 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:06,960 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:06,962 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:06,962 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:06,962 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:06,962 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:06,962 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:55:06,983 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907c7403d73f3b74c5cbb309d5228779508 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907c7403d73f3b74c5cbb309d5228779508 2023-03-08 18:55:06,990 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/77be267570e94d018e8d93015fa546d0 2023-03-08 18:55:06,994 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/77be267570e94d018e8d93015fa546d0 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/77be267570e94d018e8d93015fa546d0 2023-03-08 18:55:06,999 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/77be267570e94d018e8d93015fa546d0, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 18:55:07,000 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 38ms, sequenceid=37, compaction requested=true 2023-03-08 18:55:07,000 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:07,000 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:07,000 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:07,000 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:07,001 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,001 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:07,001 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:07,002 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:07,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:07,002 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:07,003 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,003 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:07,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:07,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:07,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:07,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@504290cb[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:07,004 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:07,004 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,006 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,006 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:07,006 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:07,007 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:07,006 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,007 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:07,008 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,008 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:07,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:07,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:07,008 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:07,008 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:07,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:07,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:07,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:07,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:07,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:07,010 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:07,010 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:07,010 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:07,010 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,010 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:07,012 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:07,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:07,012 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,012 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:07,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:07,012 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:07,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:07,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:07,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,013 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:07,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,015 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,015 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:07,016 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:07,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:07,016 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,016 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:07,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:07,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:07,016 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:07,016 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:07,016 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,016 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:07,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:07,016 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:07,016 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,017 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 18:55:07,116 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:07,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:07,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:07,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:07,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,122 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:07,122 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:07,122 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 18:55:07,122 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 18:55:07,123 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,123 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,125 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,125 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:07,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:07,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:07,125 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,125 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:07,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:07,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 18:55:07,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,126 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 18:55:07,126 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:07,126 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 18:55:07,127 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:55:07,127 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:07,127 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:07,127 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. started... 2023-03-08 18:55:07,127 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fb8791aea8f6b5c60c5c17523aa003f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:55:07,539 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120967bf09ccf8142e7bfd24d5fecd7b317 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120967bf09ccf8142e7bfd24d5fecd7b317 2023-03-08 18:55:07,944 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/46904eeefdf9428d8b94aa11790a00d4 2023-03-08 18:55:07,949 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/46904eeefdf9428d8b94aa11790a00d4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/46904eeefdf9428d8b94aa11790a00d4 2023-03-08 18:55:07,953 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/46904eeefdf9428d8b94aa11790a00d4, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 18:55:07,954 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fb8791aea8f6b5c60c5c17523aa003f3 in 827ms, sequenceid=41, compaction requested=true 2023-03-08 18:55:07,954 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:07,954 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:07,954 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:55:07,954 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:55:07,954 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,954 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 18:55:07,954 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 18:55:07,956 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,956 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:07,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:07,956 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,956 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:07,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:07,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:07,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:07,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 18:55:07,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c428ceb[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:07,958 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 18:55:07,958 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,959 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,960 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 18:55:07,960 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 18:55:07,960 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,960 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:07,960 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:07,961 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 18:55:07,961 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:07,961 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,961 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 18:55:07,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:07,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:07,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:07,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 18:55:07,964 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 18:55:07,964 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:07,964 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:07,964 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,964 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:07,966 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,966 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:07,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:07,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,966 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:07,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:07,967 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:07,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,968 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,970 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,970 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:07,970 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,970 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:07,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:07,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:07,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 18:55:07,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:07,970 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,970 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:07,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 18:55:07,970 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:07,971 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,971 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,971 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:07,971 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:07,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:07,971 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:08,071 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:08,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:08,073 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:08,075 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:08,075 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:08,076 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302308076 2023-03-08 18:55:08,076 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:55:08,077 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:55:08,078 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 18:55:08,078 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:08,079 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x5f08da05 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:55:08,083 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2175d42, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:55:08,083 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:08,917 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038fed9291501048eca0ff153a88ac3935 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601038fed9291501048eca0ff153a88ac3935 2023-03-08 18:55:08,918 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:08,931 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:55:08,932 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:43398, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:55:08,939 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:55:08,940 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:37344, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:55:08,947 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/49776efb073b4228a89847ab05cc34e4 first=Optional[r3] last=Optional[r6] 2023-03-08 18:55:08,948 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/49776efb073b4228a89847ab05cc34e4}] 2023-03-08 18:55:08,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/49776efb073b4228a89847ab05cc34e4 for inclusion in fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:08,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 18:55:08,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:55:08,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:08,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/49776efb073b4228a89847ab05cc34e4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r3ubs2vc82qhah90e5ua69nalr9kkoj5mee267tb2rnhsmh3491ujjdlog0iusvv/family1/49776efb073b4228a89847ab05cc34e4 2023-03-08 18:55:08,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r3ubs2vc82qhah90e5ua69nalr9kkoj5mee267tb2rnhsmh3491ujjdlog0iusvv/family1/49776efb073b4228a89847ab05cc34e4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/5472339c7c424142b08dac502dbfcbcb_SeqId_44_ 2023-03-08 18:55:08,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r3ubs2vc82qhah90e5ua69nalr9kkoj5mee267tb2rnhsmh3491ujjdlog0iusvv/family1/49776efb073b4228a89847ab05cc34e4 into fb8791aea8f6b5c60c5c17523aa003f3/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/5472339c7c424142b08dac502dbfcbcb_SeqId_44_ - updating store file list. 2023-03-08 18:55:08,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/5472339c7c424142b08dac502dbfcbcb_SeqId_44_ into fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:08,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r3ubs2vc82qhah90e5ua69nalr9kkoj5mee267tb2rnhsmh3491ujjdlog0iusvv/family1/49776efb073b4228a89847ab05cc34e4 into fb8791aea8f6b5c60c5c17523aa003f3/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/5472339c7c424142b08dac502dbfcbcb_SeqId_44_) 2023-03-08 18:55:08,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r3ubs2vc82qhah90e5ua69nalr9kkoj5mee267tb2rnhsmh3491ujjdlog0iusvv/family1/49776efb073b4228a89847ab05cc34e4 2023-03-08 18:55:08,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 18:55:08,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 18:55:08,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): fb8791aea8f6b5c60c5c17523aa003f3/family1 is initiating minor compaction 2023-03-08 18:55:08,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5abc0d96; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:55:08,967 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fb8791aea8f6b5c60c5c17523aa003f3/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:08,967 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/3a819bf1c4fb4cb1b80b525ccfbc11d2, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/c7e8eee14f3c46e9a44fdd97d5f79cbb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/55a6df29d30d40b9844c3b2bcc32e34b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/d9537e83994a4bd4bdf9993173fe3439, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/2d106608bb8f4174afca4a883bcf8493, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f2b0e4800ce8445bab9b842d43fd7ed9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/335d7ce1024c44408d82e0cce687db8b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/77be267570e94d018e8d93015fa546d0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/46904eeefdf9428d8b94aa11790a00d4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/5472339c7c424142b08dac502dbfcbcb_SeqId_44_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp, totalSize=50.3 K 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 3a819bf1c4fb4cb1b80b525ccfbc11d2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting c7e8eee14f3c46e9a44fdd97d5f79cbb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 55a6df29d30d40b9844c3b2bcc32e34b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting d9537e83994a4bd4bdf9993173fe3439, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 2d106608bb8f4174afca4a883bcf8493, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting f2b0e4800ce8445bab9b842d43fd7ed9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 335d7ce1024c44408d82e0cce687db8b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 77be267570e94d018e8d93015fa546d0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 46904eeefdf9428d8b94aa11790a00d4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 18:55:08,968 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 5472339c7c424142b08dac502dbfcbcb_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 18:55:08,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:08,970 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:08,972 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512288ce5915ef8c44f7db55afa49da2284f4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512288ce5915ef8c44f7db55afa49da2284f4 2023-03-08 18:55:08,973 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123138de077309954e0aa457cb365a9c5ee9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123138de077309954e0aa457cb365a9c5ee9 2023-03-08 18:55:08,974 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018745ee8c87464289814431a21763e9df to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018745ee8c87464289814431a21763e9df 2023-03-08 18:55:08,975 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103dbae00097e5f4e9b8946af8438446506 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103dbae00097e5f4e9b8946af8438446506 2023-03-08 18:55:08,975 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-08 18:55:08,975 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 18:55:08,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:08,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:08,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:08,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:08,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:08,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:08,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:08,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:09,024 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): fb8791aea8f6b5c60c5c17523aa003f3#family1#compaction#73 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:09,047 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c73ab6e66ad746d4aad5c4c9df896d64 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205c73ab6e66ad746d4aad5c4c9df896d64 2023-03-08 18:55:09,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:09,074 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ab137e2bc2414095ae054f7fd0a51efc first=Optional[r00] last=Optional[r2] 2023-03-08 18:55:09,075 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ab137e2bc2414095ae054f7fd0a51efc}] 2023-03-08 18:55:09,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ab137e2bc2414095ae054f7fd0a51efc for inclusion in fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:09,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 18:55:09,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:55:09,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:09,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ab137e2bc2414095ae054f7fd0a51efc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__h3h9fbm6d9tn424t1id9cpc0ptqdkvldm672p4ev4o7s8po92rjrf00hphffejem/family1/ab137e2bc2414095ae054f7fd0a51efc 2023-03-08 18:55:09,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__h3h9fbm6d9tn424t1id9cpc0ptqdkvldm672p4ev4o7s8po92rjrf00hphffejem/family1/ab137e2bc2414095ae054f7fd0a51efc as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_ 2023-03-08 18:55:09,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__h3h9fbm6d9tn424t1id9cpc0ptqdkvldm672p4ev4o7s8po92rjrf00hphffejem/family1/ab137e2bc2414095ae054f7fd0a51efc into fb8791aea8f6b5c60c5c17523aa003f3/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_ - updating store file list. 2023-03-08 18:55:09,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_ into fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:09,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__h3h9fbm6d9tn424t1id9cpc0ptqdkvldm672p4ev4o7s8po92rjrf00hphffejem/family1/ab137e2bc2414095ae054f7fd0a51efc into fb8791aea8f6b5c60c5c17523aa003f3/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_) 2023-03-08 18:55:09,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__h3h9fbm6d9tn424t1id9cpc0ptqdkvldm672p4ev4o7s8po92rjrf00hphffejem/family1/ab137e2bc2414095ae054f7fd0a51efc 2023-03-08 18:55:09,088 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:09,089 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:09,090 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130337f0f1627aa4ff9bbd3864a380da010 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130337f0f1627aa4ff9bbd3864a380da010 2023-03-08 18:55:09,091 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201bf3aca91571e463d8caade8c5d5aa4fb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201bf3aca91571e463d8caade8c5d5aa4fb 2023-03-08 18:55:09,092 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a5adc98e040249c6867f16dcbbd3d14d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a5adc98e040249c6867f16dcbbd3d14d 2023-03-08 18:55:09,092 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-08 18:55:09,092 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 18:55:09,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:09,092 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:09,092 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:55:09,093 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:09,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.0190 sec 2023-03-08 18:55:09,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$12(2454): Client=jenkins//172.31.2.20 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 18:55:09,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:09,584 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 18:55:09,585 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-08 18:55:09,585 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 18:55:09,587 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-08 18:55:09,590 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, REOPEN/MOVE}] 2023-03-08 18:55:09,591 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, REOPEN/MOVE 2023-03-08 18:55:09,591 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:09,591 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301709591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301709591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301709591"}]},"ts":"1678301709591"} 2023-03-08 18:55:09,593 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:09,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:09,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing fb8791aea8f6b5c60c5c17523aa003f3, disabling compactions & flushes 2023-03-08 18:55:09,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:09,846 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/eacde5d469c547c6a3767cd3d7b1ac52 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/eacde5d469c547c6a3767cd3d7b1ac52 2023-03-08 18:55:09,850 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in fb8791aea8f6b5c60c5c17523aa003f3/family1 of fb8791aea8f6b5c60c5c17523aa003f3 into eacde5d469c547c6a3767cd3d7b1ac52(size=6.8 K), total size for store is 388.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:55:09,850 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:09,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:09,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:09,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. after waiting 0 ms 2023-03-08 18:55:09,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:09,850 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678301708967; duration=0sec 2023-03-08 18:55:09,850 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:55:09,861 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/3a819bf1c4fb4cb1b80b525ccfbc11d2, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/c7e8eee14f3c46e9a44fdd97d5f79cbb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/55a6df29d30d40b9844c3b2bcc32e34b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/d9537e83994a4bd4bdf9993173fe3439, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/2d106608bb8f4174afca4a883bcf8493, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f2b0e4800ce8445bab9b842d43fd7ed9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/335d7ce1024c44408d82e0cce687db8b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/77be267570e94d018e8d93015fa546d0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/46904eeefdf9428d8b94aa11790a00d4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/5472339c7c424142b08dac502dbfcbcb_SeqId_44_] to archive 2023-03-08 18:55:09,862 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:09,864 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/3a819bf1c4fb4cb1b80b525ccfbc11d2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/3a819bf1c4fb4cb1b80b525ccfbc11d2 2023-03-08 18:55:09,865 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/c7e8eee14f3c46e9a44fdd97d5f79cbb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/c7e8eee14f3c46e9a44fdd97d5f79cbb 2023-03-08 18:55:09,866 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/55a6df29d30d40b9844c3b2bcc32e34b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/55a6df29d30d40b9844c3b2bcc32e34b 2023-03-08 18:55:09,867 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/d9537e83994a4bd4bdf9993173fe3439 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/d9537e83994a4bd4bdf9993173fe3439 2023-03-08 18:55:09,868 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/2d106608bb8f4174afca4a883bcf8493 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/2d106608bb8f4174afca4a883bcf8493 2023-03-08 18:55:09,870 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f2b0e4800ce8445bab9b842d43fd7ed9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f2b0e4800ce8445bab9b842d43fd7ed9 2023-03-08 18:55:09,871 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/335d7ce1024c44408d82e0cce687db8b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/335d7ce1024c44408d82e0cce687db8b 2023-03-08 18:55:09,872 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/77be267570e94d018e8d93015fa546d0 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/77be267570e94d018e8d93015fa546d0 2023-03-08 18:55:09,872 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/46904eeefdf9428d8b94aa11790a00d4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/46904eeefdf9428d8b94aa11790a00d4 2023-03-08 18:55:09,873 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/5472339c7c424142b08dac502dbfcbcb_SeqId_44_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/5472339c7c424142b08dac502dbfcbcb_SeqId_44_ 2023-03-08 18:55:09,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-08 18:55:09,880 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:09,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:09,880 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(3509): Not adding moved region record: fb8791aea8f6b5c60c5c17523aa003f3 to self. 2023-03-08 18:55:09,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:09,882 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=CLOSED 2023-03-08 18:55:09,882 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301709882"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301709882"}]},"ts":"1678301709882"} 2023-03-08 18:55:09,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-08 18:55:09,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627 in 290 msec 2023-03-08 18:55:09,884 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=true 2023-03-08 18:55:10,035 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:10,035 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301710035"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301710035"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301710035"}]},"ts":"1678301710035"} 2023-03-08 18:55:10,036 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:10,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:10,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fb8791aea8f6b5c60c5c17523aa003f3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:55:10,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:10,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:10,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:10,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:10,192 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:10,193 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:10,193 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:10,194 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb8791aea8f6b5c60c5c17523aa003f3 columnFamilyName family1 2023-03-08 18:55:10,198 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/38b10b4b1bce4551889e68f6c47345a3 2023-03-08 18:55:10,202 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/eacde5d469c547c6a3767cd3d7b1ac52 2023-03-08 18:55:10,204 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_ 2023-03-08 18:55:10,205 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(310): Store=fb8791aea8f6b5c60c5c17523aa003f3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:10,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:10,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:10,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:10,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened fb8791aea8f6b5c60c5c17523aa003f3; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66227516, jitterRate=-0.013133108615875244}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:55:10,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:10,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3., pid=63, masterSystemTime=1678301710188 2023-03-08 18:55:10,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:55:10,210 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 18:55:10,212 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 18:55:10,212 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1912): fb8791aea8f6b5c60c5c17523aa003f3/family1 is initiating minor compaction (all files) 2023-03-08 18:55:10,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:10,212 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fb8791aea8f6b5c60c5c17523aa003f3/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:10,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:10,212 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/38b10b4b1bce4551889e68f6c47345a3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/eacde5d469c547c6a3767cd3d7b1ac52, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp, totalSize=388.1 K 2023-03-08 18:55:10,212 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:10,213 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301710212"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301710212"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301710212"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301710212"}]},"ts":"1678301710212"} 2023-03-08 18:55:10,213 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 38b10b4b1bce4551889e68f6c47345a3, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845940 2023-03-08 18:55:10,213 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting eacde5d469c547c6a3767cd3d7b1ac52, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245940 2023-03-08 18:55:10,213 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting f025b5f09f694282b58ea5a856aae6f7_SeqId_46_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878845940 2023-03-08 18:55:10,215 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-08 18:55:10,215 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627 in 178 msec 2023-03-08 18:55:10,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-08 18:55:10,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, REOPEN/MOVE in 625 msec 2023-03-08 18:55:10,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-08 18:55:10,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 630 msec 2023-03-08 18:55:10,221 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0450 sec 2023-03-08 18:55:10,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-08 18:55:10,223 INFO [Listener at localhost/36281] compactions.TestMobCompactor(278): alter status finished 2023-03-08 18:55:10,223 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:10,224 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:10,225 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:10,226 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:55:10,226 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302310226 2023-03-08 18:55:10,228 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:55:10,229 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 18:55:10,229 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:10,229 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x08c56594 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:55:10,233 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): fb8791aea8f6b5c60c5c17523aa003f3#family1#compaction#74 average throughput is 13.82 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:10,242 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b450151, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:55:10,242 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:10,242 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:10,243 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:10,243 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:10,243 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:11,065 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128cf38887ba5394f2b8b2f5da26eb2f88e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128cf38887ba5394f2b8b2f5da26eb2f88e 2023-03-08 18:55:11,066 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:11,085 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:55:11,086 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:43410, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:55:11,093 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:55:11,095 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:37346, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:55:11,104 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4aafec536d17459a962a4a76053e96de first=Optional[r05] last=Optional[r8] 2023-03-08 18:55:11,105 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=50 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4aafec536d17459a962a4a76053e96de}] 2023-03-08 18:55:11,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4aafec536d17459a962a4a76053e96de for inclusion in fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:11,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 18:55:11,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:55:11,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:11,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4aafec536d17459a962a4a76053e96de to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2e5ihjur0baqmptnbt18ks4ujvmf2i6p7h2evsoupjhdjaic2dbfp1vd66kssdkl/family1/4aafec536d17459a962a4a76053e96de 2023-03-08 18:55:11,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2e5ihjur0baqmptnbt18ks4ujvmf2i6p7h2evsoupjhdjaic2dbfp1vd66kssdkl/family1/4aafec536d17459a962a4a76053e96de as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/58a4fefc21214707af7fe54b6160fd2f_SeqId_52_ 2023-03-08 18:55:11,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2e5ihjur0baqmptnbt18ks4ujvmf2i6p7h2evsoupjhdjaic2dbfp1vd66kssdkl/family1/4aafec536d17459a962a4a76053e96de into fb8791aea8f6b5c60c5c17523aa003f3/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/58a4fefc21214707af7fe54b6160fd2f_SeqId_52_ - updating store file list. 2023-03-08 18:55:11,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/58a4fefc21214707af7fe54b6160fd2f_SeqId_52_ into fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:11,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2e5ihjur0baqmptnbt18ks4ujvmf2i6p7h2evsoupjhdjaic2dbfp1vd66kssdkl/family1/4aafec536d17459a962a4a76053e96de into fb8791aea8f6b5c60c5c17523aa003f3/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/58a4fefc21214707af7fe54b6160fd2f_SeqId_52_) 2023-03-08 18:55:11,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2e5ihjur0baqmptnbt18ks4ujvmf2i6p7h2evsoupjhdjaic2dbfp1vd66kssdkl/family1/4aafec536d17459a962a4a76053e96de 2023-03-08 18:55:11,119 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:11,119 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:11,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120967bf09ccf8142e7bfd24d5fecd7b317 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120967bf09ccf8142e7bfd24d5fecd7b317 2023-03-08 18:55:11,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511286f67f2c98ef945deac81fe2440cd4c0d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511286f67f2c98ef945deac81fe2440cd4c0d 2023-03-08 18:55:11,122 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-08 18:55:11,122 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 18:55:11,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:11,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:11,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:11,122 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:11,122 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:55:11,123 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:11,124 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 899 msec 2023-03-08 18:55:11,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$12(2454): Client=jenkins//172.31.2.20 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-08 18:55:11,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:11,455 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/.tmp/family1/b763706e47c8494fb818dc6616969d35 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/b763706e47c8494fb818dc6616969d35 2023-03-08 18:55:11,460 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in fb8791aea8f6b5c60c5c17523aa003f3/family1 of fb8791aea8f6b5c60c5c17523aa003f3 into b763706e47c8494fb818dc6616969d35(size=192.0 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:11,460 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:11,460 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3., storeName=fb8791aea8f6b5c60c5c17523aa003f3/family1, priority=13, startTime=1678301710210; duration=1sec 2023-03-08 18:55:11,460 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:55:11,484 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/38b10b4b1bce4551889e68f6c47345a3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/eacde5d469c547c6a3767cd3d7b1ac52, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_] to archive 2023-03-08 18:55:11,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:11,487 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/38b10b4b1bce4551889e68f6c47345a3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/38b10b4b1bce4551889e68f6c47345a3 2023-03-08 18:55:11,488 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/eacde5d469c547c6a3767cd3d7b1ac52 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/eacde5d469c547c6a3767cd3d7b1ac52 2023-03-08 18:55:11,489 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/f025b5f09f694282b58ea5a856aae6f7_SeqId_46_ 2023-03-08 18:55:11,583 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-08 18:55:11,584 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 18:55:11,584 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-08 18:55:11,585 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-08 18:55:11,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, REOPEN/MOVE}] 2023-03-08 18:55:11,588 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, REOPEN/MOVE 2023-03-08 18:55:11,588 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:11,589 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301711588"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301711588"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301711588"}]},"ts":"1678301711588"} 2023-03-08 18:55:11,590 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:11,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:11,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing fb8791aea8f6b5c60c5c17523aa003f3, disabling compactions & flushes 2023-03-08 18:55:11,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:11,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:11,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. after waiting 0 ms 2023-03-08 18:55:11,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:11,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-03-08 18:55:11,746 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:11,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:11,746 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(3509): Not adding moved region record: fb8791aea8f6b5c60c5c17523aa003f3 to self. 2023-03-08 18:55:11,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:11,747 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=CLOSED 2023-03-08 18:55:11,748 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301711747"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301711747"}]},"ts":"1678301711747"} 2023-03-08 18:55:11,750 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-08 18:55:11,750 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627 in 159 msec 2023-03-08 18:55:11,750 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=true 2023-03-08 18:55:11,900 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:11,901 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301711900"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301711900"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301711900"}]},"ts":"1678301711900"} 2023-03-08 18:55:11,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:12,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:12,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fb8791aea8f6b5c60c5c17523aa003f3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:55:12,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:12,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:12,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:12,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:12,058 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:12,059 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:12,059 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1 2023-03-08 18:55:12,060 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb8791aea8f6b5c60c5c17523aa003f3 columnFamilyName family1 2023-03-08 18:55:12,064 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/58a4fefc21214707af7fe54b6160fd2f_SeqId_52_ 2023-03-08 18:55:12,067 DEBUG [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/family1/b763706e47c8494fb818dc6616969d35 2023-03-08 18:55:12,067 INFO [StoreOpener-fb8791aea8f6b5c60c5c17523aa003f3-1] regionserver.HStore(310): Store=fb8791aea8f6b5c60c5c17523aa003f3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:12,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:12,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:12,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:55:12,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened fb8791aea8f6b5c60c5c17523aa003f3; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70371314, jitterRate=0.048614293336868286}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:55:12,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:55:12,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3., pid=69, masterSystemTime=1678301712054 2023-03-08 18:55:12,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:12,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:55:12,074 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=fb8791aea8f6b5c60c5c17523aa003f3, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:12,074 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678301712074"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301712074"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301712074"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301712074"}]},"ts":"1678301712074"} 2023-03-08 18:55:12,077 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-08 18:55:12,077 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure fb8791aea8f6b5c60c5c17523aa003f3, server=jenkins-hbase6.apache.org,46397,1678301621627 in 173 msec 2023-03-08 18:55:12,078 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-08 18:55:12,078 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fb8791aea8f6b5c60c5c17523aa003f3, REOPEN/MOVE in 490 msec 2023-03-08 18:55:12,079 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-08 18:55:12,079 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 493 msec 2023-03-08 18:55:12,081 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 906 msec 2023-03-08 18:55:12,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-08 18:55:12,082 INFO [Listener at localhost/36281] compactions.TestMobCompactor(278): alter status finished 2023-03-08 18:55:12,083 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 18:55:12,084 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:12,085 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:12,085 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:55:12,085 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302312085 2023-03-08 18:55:12,086 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:55:12,086 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 18:55:12,086 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:12,087 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x7794ab91 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:55:12,091 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7cce0ce6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:55:12,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:12,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:12,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:12,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:12,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:12,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:12,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:12,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 18:55:12,092 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 18:55:12,092 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:55:12,093 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 18:55:12,094 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 9 msec 2023-03-08 18:55:12,152 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=249 (was 242) - Thread LEAK? -, OpenFileDescriptor=642 (was 611) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=154 (was 170), ProcessCount=172 (was 172), AvailableMemoryMB=7042 (was 7057) 2023-03-08 18:55:12,164 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=249, OpenFileDescriptor=642, MaxFileDescriptor=60000, SystemLoadAverage=154, ProcessCount=172, AvailableMemoryMB=7041 2023-03-08 18:55:12,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 18:55:12,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-08 18:55:12,170 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:55:12,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-08 18:55:12,170 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:55:12,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 18:55:12,172 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:12,172 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca empty. 2023-03-08 18:55:12,173 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:12,173 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-08 18:55:12,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 18:55:12,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 18:55:12,581 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-08 18:55:12,582 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 68cb8694da4460d96837a13fc4466eca, NAME => 'testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:55:12,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 18:55:12,988 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:12,988 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 68cb8694da4460d96837a13fc4466eca, disabling compactions & flushes 2023-03-08 18:55:12,988 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:12,988 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:12,988 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. after waiting 0 ms 2023-03-08 18:55:12,988 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:12,988 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:12,988 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 68cb8694da4460d96837a13fc4466eca: 2023-03-08 18:55:12,990 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:55:12,991 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678301712991"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301712991"}]},"ts":"1678301712991"} 2023-03-08 18:55:12,992 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:55:12,993 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:55:12,993 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301712993"}]},"ts":"1678301712993"} 2023-03-08 18:55:12,994 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-08 18:55:12,997 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=68cb8694da4460d96837a13fc4466eca, ASSIGN}] 2023-03-08 18:55:12,999 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=68cb8694da4460d96837a13fc4466eca, ASSIGN 2023-03-08 18:55:12,999 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=68cb8694da4460d96837a13fc4466eca, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:55:13,150 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=68cb8694da4460d96837a13fc4466eca, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:13,150 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678301713150"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301713150"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301713150"}]},"ts":"1678301713150"} 2023-03-08 18:55:13,152 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 68cb8694da4460d96837a13fc4466eca, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:13,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 18:55:13,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:13,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 68cb8694da4460d96837a13fc4466eca, NAME => 'testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:55:13,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-08 18:55:13,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-08 18:55:13,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:13,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:13,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:13,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:13,310 INFO [StoreOpener-68cb8694da4460d96837a13fc4466eca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:13,311 DEBUG [StoreOpener-68cb8694da4460d96837a13fc4466eca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1 2023-03-08 18:55:13,311 DEBUG [StoreOpener-68cb8694da4460d96837a13fc4466eca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1 2023-03-08 18:55:13,312 INFO [StoreOpener-68cb8694da4460d96837a13fc4466eca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68cb8694da4460d96837a13fc4466eca columnFamilyName f1 2023-03-08 18:55:13,313 INFO [StoreOpener-68cb8694da4460d96837a13fc4466eca-1] regionserver.HStore(310): Store=68cb8694da4460d96837a13fc4466eca/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:13,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:13,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:13,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:13,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:55:13,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 68cb8694da4460d96837a13fc4466eca; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74944181, jitterRate=0.11675532162189484}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:55:13,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 68cb8694da4460d96837a13fc4466eca: 2023-03-08 18:55:13,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca., pid=73, masterSystemTime=1678301713303 2023-03-08 18:55:13,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:13,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:13,321 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=68cb8694da4460d96837a13fc4466eca, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:13,321 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678301713321"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301713321"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301713321"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301713321"}]},"ts":"1678301713321"} 2023-03-08 18:55:13,323 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-08 18:55:13,323 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 68cb8694da4460d96837a13fc4466eca, server=jenkins-hbase6.apache.org,46397,1678301621627 in 170 msec 2023-03-08 18:55:13,325 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-08 18:55:13,325 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=68cb8694da4460d96837a13fc4466eca, ASSIGN in 326 msec 2023-03-08 18:55:13,325 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:55:13,325 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301713325"}]},"ts":"1678301713325"} 2023-03-08 18:55:13,326 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-08 18:55:13,329 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:55:13,330 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1610 sec 2023-03-08 18:55:13,617 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-08 18:55:14,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 18:55:14,276 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-08 18:55:14,276 DEBUG [Listener at localhost/36281] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-08 18:55:14,278 INFO [Listener at localhost/36281] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-08 18:55:14,280 INFO [Listener at localhost/36281] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-08 18:55:14,281 INFO [Listener at localhost/36281] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-08 18:55:14,281 INFO [Listener at localhost/36281] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-08 18:55:14,285 INFO [Listener at localhost/36281] regionserver.HRegion(2745): Flushing 68cb8694da4460d96837a13fc4466eca 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-08 18:55:14,300 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303080b1ecf72e9064f6d8fde46a277103e44 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303080b1ecf72e9064f6d8fde46a277103e44 2023-03-08 18:55:14,705 INFO [Listener at localhost/36281] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/.tmp/f1/50365381dcf34af88caf8952705ac8b4 2023-03-08 18:55:14,709 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/.tmp/f1/50365381dcf34af88caf8952705ac8b4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4 2023-03-08 18:55:14,713 INFO [Listener at localhost/36281] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4, entries=1, sequenceid=5, filesize=4.9 K 2023-03-08 18:55:14,714 INFO [Listener at localhost/36281] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 68cb8694da4460d96837a13fc4466eca in 429ms, sequenceid=5, compaction requested=false 2023-03-08 18:55:14,714 DEBUG [Listener at localhost/36281] regionserver.HRegion(2446): Flush status journal for 68cb8694da4460d96837a13fc4466eca: 2023-03-08 18:55:14,716 INFO [Listener at localhost/36281] regionserver.HRegion(2745): Flushing 68cb8694da4460d96837a13fc4466eca 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-08 18:55:15,127 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030814737784830848bcb6a0193dff2dfe80 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030814737784830848bcb6a0193dff2dfe80 2023-03-08 18:55:15,427 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:55:15,533 INFO [Listener at localhost/36281] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/.tmp/f1/a866710345604c0aa018651f31caedd7 2023-03-08 18:55:15,537 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/.tmp/f1/a866710345604c0aa018651f31caedd7 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7 2023-03-08 18:55:15,540 INFO [Listener at localhost/36281] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7, entries=1, sequenceid=9, filesize=4.9 K 2023-03-08 18:55:15,541 INFO [Listener at localhost/36281] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 68cb8694da4460d96837a13fc4466eca in 825ms, sequenceid=9, compaction requested=false 2023-03-08 18:55:15,541 DEBUG [Listener at localhost/36281] regionserver.HRegion(2446): Flush status journal for 68cb8694da4460d96837a13fc4466eca: 2023-03-08 18:55:15,542 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-08 18:55:15,543 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 18:55:15,544 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 18:55:15,545 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:55:15,545 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302315545 2023-03-08 18:55:15,545 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 18:55:15,545 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-08 18:55:15,545 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-08 18:55:15,546 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x6b66c18e to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:55:15,550 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67c35938, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:55:15,550 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testGetAfterCompaction' and column='f1' 2023-03-08 18:55:15,594 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,594 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,644 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,645 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,695 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,695 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,746 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,747 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,797 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,798 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,848 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,848 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,899 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,899 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,950 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:15,950 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,001 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,001 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,051 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,052 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,102 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,102 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,153 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,153 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,204 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,204 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,255 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,255 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,305 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,306 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,356 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,356 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,372 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030802910d994e154c2384950f9314f12a7a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030802910d994e154c2384950f9314f12a7a 2023-03-08 18:55:16,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testGetAfterCompaction' column='f1' 2023-03-08 18:55:16,386 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:55:16,387 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:34950, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:55:16,392 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:55:16,394 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:47142, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:55:16,401 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/d172021e13f1434790d93d0b0d9ce34d first=Optional[r1] last=Optional[r2] 2023-03-08 18:55:16,402 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{f1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/d172021e13f1434790d93d0b0d9ce34d}] 2023-03-08 18:55:16,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/d172021e13f1434790d93d0b0d9ce34d for inclusion in 68cb8694da4460d96837a13fc4466eca/f1 2023-03-08 18:55:16,407 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,407 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-08 18:55:16,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:55:16,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 68cb8694da4460d96837a13fc4466eca: 2023-03-08 18:55:16,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/d172021e13f1434790d93d0b0d9ce34d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testGetAfterCompaction__6dj4utnh7fpniqfuqdlilkmh4uqirl3bm4gpofbugnsgoh0lj5d8e6pk30s6e62t/f1/d172021e13f1434790d93d0b0d9ce34d 2023-03-08 18:55:16,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testGetAfterCompaction__6dj4utnh7fpniqfuqdlilkmh4uqirl3bm4gpofbugnsgoh0lj5d8e6pk30s6e62t/f1/d172021e13f1434790d93d0b0d9ce34d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_ 2023-03-08 18:55:16,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testGetAfterCompaction__6dj4utnh7fpniqfuqdlilkmh4uqirl3bm4gpofbugnsgoh0lj5d8e6pk30s6e62t/f1/d172021e13f1434790d93d0b0d9ce34d into 68cb8694da4460d96837a13fc4466eca/f1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_ - updating store file list. 2023-03-08 18:55:16,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_ into 68cb8694da4460d96837a13fc4466eca/f1 2023-03-08 18:55:16,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testGetAfterCompaction__6dj4utnh7fpniqfuqdlilkmh4uqirl3bm4gpofbugnsgoh0lj5d8e6pk30s6e62t/f1/d172021e13f1434790d93d0b0d9ce34d into 68cb8694da4460d96837a13fc4466eca/f1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_) 2023-03-08 18:55:16,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testGetAfterCompaction__6dj4utnh7fpniqfuqdlilkmh4uqirl3bm4gpofbugnsgoh0lj5d8e6pk30s6e62t/f1/d172021e13f1434790d93d0b0d9ce34d 2023-03-08 18:55:16,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(1912): 68cb8694da4460d96837a13fc4466eca/f1 is initiating minor compaction 2023-03-08 18:55:16,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@548ab2da; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:55:16,415 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 68cb8694da4460d96837a13fc4466eca/f1 in testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:16,415 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/.tmp, totalSize=10.0 K 2023-03-08 18:55:16,416 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting a866710345604c0aa018651f31caedd7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 18:55:16,416 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 02172b48e75e4279b912d0d062a3f6bb_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 18:55:16,417 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testGetAfterCompaction' column='f1' 2023-03-08 18:55:16,418 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:16,420 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303080b1ecf72e9064f6d8fde46a277103e44 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303080b1ecf72e9064f6d8fde46a277103e44 2023-03-08 18:55:16,422 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030814737784830848bcb6a0193dff2dfe80 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030814737784830848bcb6a0193dff2dfe80 2023-03-08 18:55:16,422 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-08 18:55:16,422 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 68cb8694da4460d96837a13fc4466eca#f1#compaction#77 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:16,423 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 18:55:16,423 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testGetAfterCompaction' and column='f1' 2023-03-08 18:55:16,423 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-08 18:55:16,423 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:55:16,424 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 18:55:16,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 881 msec 2023-03-08 18:55:16,458 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,458 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,458 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,462 DEBUG [Listener at localhost/36281] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 18:55:16,466 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:34964, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 18:55:16,467 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46397] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:16,467 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-08 18:55:16,468 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 18:55:16,468 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46397] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 18:55:16,468 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46397] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. because compaction request was cancelled 2023-03-08 18:55:16,520 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,520 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,520 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,571 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,571 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,571 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,622 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,622 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,622 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,673 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,673 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,673 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,724 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,724 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,724 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,775 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,775 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,775 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,826 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,826 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,826 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,877 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,877 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,877 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,928 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,928 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,929 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,979 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:16,979 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:16,980 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,030 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:17,031 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,031 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,081 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:17,082 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,082 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,132 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:17,133 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,133 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,184 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:17,184 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,184 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,235 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678301716368; access_time=1678301715559; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 18:55:17,235 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,235 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301715532; access_time=1678301714717; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,241 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/.tmp/f1/eca05484f853422f8d7c1a9aab63926a as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/eca05484f853422f8d7c1a9aab63926a 2023-03-08 18:55:17,246 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 68cb8694da4460d96837a13fc4466eca/f1 of 68cb8694da4460d96837a13fc4466eca into eca05484f853422f8d7c1a9aab63926a(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:55:17,246 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 68cb8694da4460d96837a13fc4466eca: 2023-03-08 18:55:17,246 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1678301716415; duration=0sec 2023-03-08 18:55:17,246 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:55:17,280 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_] to archive 2023-03-08 18:55:17,281 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:17,282 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/a866710345604c0aa018651f31caedd7 2023-03-08 18:55:17,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/02172b48e75e4279b912d0d062a3f6bb_SeqId_12_ 2023-03-08 18:55:17,285 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678301714704; access_time=1678301714288; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,286 INFO [Listener at localhost/36281] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/eca05484f853422f8d7c1a9aab63926a; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678301717237; access_time=1678301716421; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 18:55:17,402 INFO [Listener at localhost/36281] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-08 18:55:17,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testGetAfterCompaction 2023-03-08 18:55:17,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-08 18:55:17,414 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301717414"}]},"ts":"1678301717414"} 2023-03-08 18:55:17,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 18:55:17,415 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-08 18:55:17,417 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-08 18:55:17,418 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=68cb8694da4460d96837a13fc4466eca, UNASSIGN}] 2023-03-08 18:55:17,419 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=68cb8694da4460d96837a13fc4466eca, UNASSIGN 2023-03-08 18:55:17,420 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=68cb8694da4460d96837a13fc4466eca, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:17,420 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678301717420"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301717420"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301717420"}]},"ts":"1678301717420"} 2023-03-08 18:55:17,421 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 68cb8694da4460d96837a13fc4466eca, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:17,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 18:55:17,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close 68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:17,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 68cb8694da4460d96837a13fc4466eca, disabling compactions & flushes 2023-03-08 18:55:17,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:17,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:17,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. after waiting 0 ms 2023-03-08 18:55:17,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:17,577 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-08 18:55:17,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-08 18:55:17,579 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca. 2023-03-08 18:55:17,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 68cb8694da4460d96837a13fc4466eca: 2023-03-08 18:55:17,580 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed 68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:17,580 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=68cb8694da4460d96837a13fc4466eca, regionState=CLOSED 2023-03-08 18:55:17,581 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678301717580"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301717580"}]},"ts":"1678301717580"} 2023-03-08 18:55:17,583 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-08 18:55:17,583 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 68cb8694da4460d96837a13fc4466eca, server=jenkins-hbase6.apache.org,46397,1678301621627 in 161 msec 2023-03-08 18:55:17,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-08 18:55:17,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=68cb8694da4460d96837a13fc4466eca, UNASSIGN in 165 msec 2023-03-08 18:55:17,584 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301717584"}]},"ts":"1678301717584"} 2023-03-08 18:55:17,585 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-08 18:55:17,586 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-08 18:55:17,588 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 180 msec 2023-03-08 18:55:17,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 18:55:17,717 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-08 18:55:17,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testGetAfterCompaction 2023-03-08 18:55:17,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 18:55:17,729 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 18:55:17,730 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 18:55:17,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-08 18:55:17,733 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:17,735 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1, FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/recovered.edits] 2023-03-08 18:55:17,739 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/50365381dcf34af88caf8952705ac8b4 2023-03-08 18:55:17,740 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/eca05484f853422f8d7c1a9aab63926a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/f1/eca05484f853422f8d7c1a9aab63926a 2023-03-08 18:55:17,743 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/recovered.edits/15.seqid to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca/recovered.edits/15.seqid 2023-03-08 18:55:17,744 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testGetAfterCompaction/68cb8694da4460d96837a13fc4466eca 2023-03-08 18:55:17,744 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-08 18:55:17,744 DEBUG [PEWorker-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-08 18:55:17,745 DEBUG [PEWorker-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-08 18:55:17,748 DEBUG [PEWorker-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030802910d994e154c2384950f9314f12a7a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030802910d994e154c2384950f9314f12a7a 2023-03-08 18:55:17,748 DEBUG [PEWorker-5] backup.HFileArchiver(596): Deleted hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-08 18:55:17,750 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 18:55:17,776 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-08 18:55:17,778 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-08 18:55:17,779 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 18:55:17,780 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-08 18:55:17,780 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678301717780"}]},"ts":"9223372036854775807"} 2023-03-08 18:55:17,781 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-08 18:55:17,781 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 68cb8694da4460d96837a13fc4466eca, NAME => 'testGetAfterCompaction,,1678301712168.68cb8694da4460d96837a13fc4466eca.', STARTKEY => '', ENDKEY => ''}] 2023-03-08 18:55:17,781 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-08 18:55:17,781 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1678301717781"}]},"ts":"9223372036854775807"} 2023-03-08 18:55:17,783 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-08 18:55:17,789 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 18:55:17,791 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 67 msec 2023-03-08 18:55:17,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-08 18:55:17,832 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-08 18:55:17,845 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=255 (was 249) - Thread LEAK? -, OpenFileDescriptor=648 (was 642) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=150 (was 154), ProcessCount=172 (was 172), AvailableMemoryMB=7038 (was 7041) 2023-03-08 18:55:17,856 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=255, OpenFileDescriptor=648, MaxFileDescriptor=60000, SystemLoadAverage=150, ProcessCount=172, AvailableMemoryMB=7037 2023-03-08 18:55:17,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 18:55:17,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-08 18:55:17,859 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:55:17,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-08 18:55:17,860 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:55:17,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 18:55:17,861 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:17,861 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:17,861 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:17,862 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892 empty. 2023-03-08 18:55:17,862 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d empty. 2023-03-08 18:55:17,862 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13 empty. 2023-03-08 18:55:17,862 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:17,862 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:17,862 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:17,862 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-08 18:55:17,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 18:55:18,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 18:55:18,274 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-08 18:55:18,275 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 70855e40ae020380397c6a9b2ff4c892, NAME => 'testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:55:18,275 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 087e27baac917de6bad03c6444a74b6d, NAME => 'testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:55:18,275 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 924cd1dc6b0366a92e4bc7c5a4762a13, NAME => 'testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:55:18,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 18:55:18,682 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:18,682 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 087e27baac917de6bad03c6444a74b6d, disabling compactions & flushes 2023-03-08 18:55:18,682 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:18,682 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:18,682 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. after waiting 0 ms 2023-03-08 18:55:18,682 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:18,682 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:18,683 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:18,684 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:18,684 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 70855e40ae020380397c6a9b2ff4c892, disabling compactions & flushes 2023-03-08 18:55:18,684 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:18,684 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:18,684 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. after waiting 0 ms 2023-03-08 18:55:18,684 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:18,684 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:18,684 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:18,688 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:18,688 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 924cd1dc6b0366a92e4bc7c5a4762a13, disabling compactions & flushes 2023-03-08 18:55:18,688 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:18,688 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:18,688 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. after waiting 0 ms 2023-03-08 18:55:18,688 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:18,688 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:18,688 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:18,690 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:55:18,691 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301718691"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301718691"}]},"ts":"1678301718691"} 2023-03-08 18:55:18,692 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301718691"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301718691"}]},"ts":"1678301718691"} 2023-03-08 18:55:18,692 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301718691"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301718691"}]},"ts":"1678301718691"} 2023-03-08 18:55:18,693 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 18:55:18,694 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:55:18,694 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301718694"}]},"ts":"1678301718694"} 2023-03-08 18:55:18,695 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-08 18:55:18,700 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, ASSIGN}] 2023-03-08 18:55:18,701 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, ASSIGN 2023-03-08 18:55:18,701 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, ASSIGN 2023-03-08 18:55:18,701 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, ASSIGN 2023-03-08 18:55:18,701 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:55:18,702 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:55:18,702 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:55:18,853 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=087e27baac917de6bad03c6444a74b6d, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:18,853 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=924cd1dc6b0366a92e4bc7c5a4762a13, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:18,853 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=70855e40ae020380397c6a9b2ff4c892, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:18,854 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301718853"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301718853"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301718853"}]},"ts":"1678301718853"} 2023-03-08 18:55:18,854 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301718853"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301718853"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301718853"}]},"ts":"1678301718853"} 2023-03-08 18:55:18,854 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301718853"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301718853"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301718853"}]},"ts":"1678301718853"} 2023-03-08 18:55:18,855 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure 087e27baac917de6bad03c6444a74b6d, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:18,856 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 924cd1dc6b0366a92e4bc7c5a4762a13, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:18,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure 70855e40ae020380397c6a9b2ff4c892, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:18,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 18:55:19,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:19,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 087e27baac917de6bad03c6444a74b6d, NAME => 'testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 18:55:19,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:19,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:19,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:19,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:19,011 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:19,013 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1 2023-03-08 18:55:19,013 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1 2023-03-08 18:55:19,013 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 087e27baac917de6bad03c6444a74b6d columnFamilyName family1 2023-03-08 18:55:19,014 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(310): Store=087e27baac917de6bad03c6444a74b6d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:19,015 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:19,016 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2 2023-03-08 18:55:19,016 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2 2023-03-08 18:55:19,016 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 087e27baac917de6bad03c6444a74b6d columnFamilyName family2 2023-03-08 18:55:19,017 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(310): Store=087e27baac917de6bad03c6444a74b6d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:19,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:19,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:19,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:19,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:19,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:55:19,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 087e27baac917de6bad03c6444a74b6d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68615608, jitterRate=0.022452235221862793}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:19,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:19,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d., pid=83, masterSystemTime=1678301719007 2023-03-08 18:55:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:19,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:19,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 70855e40ae020380397c6a9b2ff4c892, NAME => 'testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 18:55:19,026 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=087e27baac917de6bad03c6444a74b6d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:19,026 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301719026"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301719026"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301719026"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301719026"}]},"ts":"1678301719026"} 2023-03-08 18:55:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:19,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:19,028 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:19,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-03-08 18:55:19,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure 087e27baac917de6bad03c6444a74b6d, server=jenkins-hbase6.apache.org,46397,1678301621627 in 172 msec 2023-03-08 18:55:19,029 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1 2023-03-08 18:55:19,029 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1 2023-03-08 18:55:19,029 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70855e40ae020380397c6a9b2ff4c892 columnFamilyName family1 2023-03-08 18:55:19,030 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, ASSIGN in 329 msec 2023-03-08 18:55:19,030 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(310): Store=70855e40ae020380397c6a9b2ff4c892/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:19,031 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:19,032 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2 2023-03-08 18:55:19,032 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2 2023-03-08 18:55:19,032 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70855e40ae020380397c6a9b2ff4c892 columnFamilyName family2 2023-03-08 18:55:19,033 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(310): Store=70855e40ae020380397c6a9b2ff4c892/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:19,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:19,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:19,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:19,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:19,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:55:19,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 70855e40ae020380397c6a9b2ff4c892; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69823631, jitterRate=0.04045318067073822}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:19,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:19,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892., pid=85, masterSystemTime=1678301719007 2023-03-08 18:55:19,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:19,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:19,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:19,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 924cd1dc6b0366a92e4bc7c5a4762a13, NAME => 'testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 18:55:19,040 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=70855e40ae020380397c6a9b2ff4c892, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:19,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:19,040 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301719040"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301719040"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301719040"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301719040"}]},"ts":"1678301719040"} 2023-03-08 18:55:19,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:19,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:19,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:19,041 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:19,043 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1 2023-03-08 18:55:19,043 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1 2023-03-08 18:55:19,043 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-03-08 18:55:19,043 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure 70855e40ae020380397c6a9b2ff4c892, server=jenkins-hbase6.apache.org,46397,1678301621627 in 180 msec 2023-03-08 18:55:19,043 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 924cd1dc6b0366a92e4bc7c5a4762a13 columnFamilyName family1 2023-03-08 18:55:19,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, ASSIGN in 345 msec 2023-03-08 18:55:19,044 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(310): Store=924cd1dc6b0366a92e4bc7c5a4762a13/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:19,045 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:19,046 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2 2023-03-08 18:55:19,046 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2 2023-03-08 18:55:19,046 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 924cd1dc6b0366a92e4bc7c5a4762a13 columnFamilyName family2 2023-03-08 18:55:19,047 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(310): Store=924cd1dc6b0366a92e4bc7c5a4762a13/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:19,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:19,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:19,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:19,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:19,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:55:19,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 924cd1dc6b0366a92e4bc7c5a4762a13; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60500888, jitterRate=-0.09846651554107666}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:19,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:19,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13., pid=84, masterSystemTime=1678301719007 2023-03-08 18:55:19,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:19,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:19,055 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=924cd1dc6b0366a92e4bc7c5a4762a13, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:19,055 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301719055"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301719055"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301719055"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301719055"}]},"ts":"1678301719055"} 2023-03-08 18:55:19,058 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-03-08 18:55:19,058 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 924cd1dc6b0366a92e4bc7c5a4762a13, server=jenkins-hbase6.apache.org,46397,1678301621627 in 201 msec 2023-03-08 18:55:19,059 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-03-08 18:55:19,059 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, ASSIGN in 360 msec 2023-03-08 18:55:19,060 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:55:19,060 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301719060"}]},"ts":"1678301719060"} 2023-03-08 18:55:19,061 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-08 18:55:19,063 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:55:19,064 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2060 sec 2023-03-08 18:55:19,618 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 18:55:19,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 18:55:19,965 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-08 18:55:19,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:19,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:19,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:19,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:19,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 18:55:19,984 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 18:55:19,984 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:19,984 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 18:55:19,984 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 18:55:19,985 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:19,985 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:19,987 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:19,987 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:19,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:19,987 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:19,987 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:19,988 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:19,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:19,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:19,988 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 18:55:19,988 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 18:55:19,988 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 18:55:19,988 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:19,988 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 18:55:19,989 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:19,989 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. started... 2023-03-08 18:55:19,989 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:19,989 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:19,989 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:19,989 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:19,989 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. started... 2023-03-08 18:55:19,989 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. started... 2023-03-08 18:55:19,989 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 924cd1dc6b0366a92e4bc7c5a4762a13 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:55:19,990 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 087e27baac917de6bad03c6444a74b6d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:55:19,991 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 70855e40ae020380397c6a9b2ff4c892 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:55:20,407 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612 2023-03-08 18:55:20,408 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323 2023-03-08 18:55:20,408 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f 2023-03-08 18:55:20,419 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/a7594ec12b084addbcc1369ec9928778 2023-03-08 18:55:20,816 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/1f9fdb228c8647fb87719f5cd2ad1bc0 2023-03-08 18:55:20,816 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/8e47fa9593bd4665acef00d8eb428147 2023-03-08 18:55:20,832 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303087d11bc475e9e40cb8c8ce3a6dd44bc19 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303087d11bc475e9e40cb8c8ce3a6dd44bc19 2023-03-08 18:55:21,168 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7794ab91 to 127.0.0.1:54098 2023-03-08 18:55:21,168 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:21,168 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:55:21,168 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b66c18e to 127.0.0.1:54098 2023-03-08 18:55:21,168 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:21,169 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:55:21,169 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x08c56594 to 127.0.0.1:54098 2023-03-08 18:55:21,169 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:21,169 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:55:21,169 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f08da05 to 127.0.0.1:54098 2023-03-08 18:55:21,169 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:21,169 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x40a99ac2 to 127.0.0.1:54098 2023-03-08 18:55:21,169 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:21,169 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:55:21,169 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x494d79f7 to 127.0.0.1:54098 2023-03-08 18:55:21,169 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:21,170 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:55:21,170 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67a79a92 to 127.0.0.1:54098 2023-03-08 18:55:21,170 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:21,230 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303089a800a8a78da4ab5b1a367f95f5f08a7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303089a800a8a78da4ab5b1a367f95f5f08a7 2023-03-08 18:55:21,230 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308d79cb2624513481c98336e55882b921d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d79cb2624513481c98336e55882b921d 2023-03-08 18:55:21,236 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/3f86ffff34c0468fb5aa1b8d80d7fc87 2023-03-08 18:55:21,237 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/cdcbd35b856741ea902048713afcc63a 2023-03-08 18:55:21,240 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/8e47fa9593bd4665acef00d8eb428147 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/8e47fa9593bd4665acef00d8eb428147 2023-03-08 18:55:21,241 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/a7594ec12b084addbcc1369ec9928778 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/a7594ec12b084addbcc1369ec9928778 2023-03-08 18:55:21,244 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/8e47fa9593bd4665acef00d8eb428147, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 18:55:21,244 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/a7594ec12b084addbcc1369ec9928778, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 18:55:21,245 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/3f86ffff34c0468fb5aa1b8d80d7fc87 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/3f86ffff34c0468fb5aa1b8d80d7fc87 2023-03-08 18:55:21,245 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/cdcbd35b856741ea902048713afcc63a as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/cdcbd35b856741ea902048713afcc63a 2023-03-08 18:55:21,248 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/3f86ffff34c0468fb5aa1b8d80d7fc87, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:55:21,249 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/cdcbd35b856741ea902048713afcc63a, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:55:21,249 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 087e27baac917de6bad03c6444a74b6d in 1259ms, sequenceid=5, compaction requested=false 2023-03-08 18:55:21,250 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:21,250 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:21,250 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:21,250 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 924cd1dc6b0366a92e4bc7c5a4762a13 in 1261ms, sequenceid=5, compaction requested=false 2023-03-08 18:55:21,250 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:21,250 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:21,250 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:21,636 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/8cf2aef7d2a74556b2cb4eb00f828dba 2023-03-08 18:55:21,640 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/1f9fdb228c8647fb87719f5cd2ad1bc0 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/1f9fdb228c8647fb87719f5cd2ad1bc0 2023-03-08 18:55:21,644 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/1f9fdb228c8647fb87719f5cd2ad1bc0, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 18:55:21,644 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/8cf2aef7d2a74556b2cb4eb00f828dba as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/8cf2aef7d2a74556b2cb4eb00f828dba 2023-03-08 18:55:21,647 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/8cf2aef7d2a74556b2cb4eb00f828dba, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:55:21,648 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 70855e40ae020380397c6a9b2ff4c892 in 1657ms, sequenceid=5, compaction requested=false 2023-03-08 18:55:21,648 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:21,648 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:21,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:21,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:21,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 18:55:21,649 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 18:55:21,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:21,650 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:21,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:21,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:21,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:21,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:21,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:21,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:21,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:21,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 18:55:21,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9ef5202[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:21,652 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 18:55:21,652 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:21,653 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:21,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:21,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:21,653 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 18:55:21,653 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,653 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:21,653 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 18:55:21,654 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:21,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,655 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 18:55:21,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:21,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:21,655 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:21,655 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 18:55:21,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:21,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:21,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:21,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 18:55:21,657 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 18:55:21,657 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:21,657 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:21,657 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 18:55:21,657 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:21,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:21,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:21,660 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:21,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,660 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:21,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:21,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:21,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:21,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:21,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:21,661 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:21,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:21,664 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 18:55:21,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:21,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:21,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:21,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:21,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:21,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:21,664 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:21,764 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:21,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:21,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:21,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:21,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:21,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:21,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:21,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 18:55:21,771 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 18:55:21,771 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:21,771 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 18:55:21,771 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 18:55:21,771 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,771 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:21,773 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:21,773 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:21,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:21,773 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:21,773 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:21,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:21,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:21,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:21,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:21,774 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 18:55:21,774 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 18:55:21,774 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 18:55:21,774 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 18:55:21,774 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:21,774 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 18:55:21,779 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:21,779 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. started... 2023-03-08 18:55:21,779 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:21,779 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. started... 2023-03-08 18:55:21,779 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 70855e40ae020380397c6a9b2ff4c892 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:55:21,779 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 924cd1dc6b0366a92e4bc7c5a4762a13 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:55:21,782 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:21,782 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:21,782 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:21,782 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. started... 2023-03-08 18:55:21,783 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 087e27baac917de6bad03c6444a74b6d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:55:22,206 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae 2023-03-08 18:55:22,207 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db 2023-03-08 18:55:22,212 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/00a3dc9d388b42da8cd5f72975734740 2023-03-08 18:55:22,213 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361 2023-03-08 18:55:22,213 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8 2023-03-08 18:55:22,217 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/c8c661ff321b4588b2eb3fea2a3e6a20 2023-03-08 18:55:22,628 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030850c9ef639bbc43b4abcec48683ce6184 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030850c9ef639bbc43b4abcec48683ce6184 2023-03-08 18:55:22,628 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303089b7f1f4647b2476694cb6c5718ae3116 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089b7f1f4647b2476694cb6c5718ae3116 2023-03-08 18:55:22,631 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308daa3430e10524dec9d5fa13a1effa3da to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308daa3430e10524dec9d5fa13a1effa3da 2023-03-08 18:55:22,642 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/a154990d75dd4a3681e935917dce1002 2023-03-08 18:55:22,645 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/2ea9681be9bb4ce8b6811c960e6d58c7 2023-03-08 18:55:22,647 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8 2023-03-08 18:55:22,649 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/c8c661ff321b4588b2eb3fea2a3e6a20 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/c8c661ff321b4588b2eb3fea2a3e6a20 2023-03-08 18:55:22,651 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 18:55:22,652 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/a154990d75dd4a3681e935917dce1002 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/a154990d75dd4a3681e935917dce1002 2023-03-08 18:55:22,652 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/c8c661ff321b4588b2eb3fea2a3e6a20, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 18:55:22,653 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/2ea9681be9bb4ce8b6811c960e6d58c7 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/2ea9681be9bb4ce8b6811c960e6d58c7 2023-03-08 18:55:22,656 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/a154990d75dd4a3681e935917dce1002, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:55:22,656 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/2ea9681be9bb4ce8b6811c960e6d58c7, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:55:22,657 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 087e27baac917de6bad03c6444a74b6d in 875ms, sequenceid=9, compaction requested=false 2023-03-08 18:55:22,657 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 70855e40ae020380397c6a9b2ff4c892 in 878ms, sequenceid=9, compaction requested=false 2023-03-08 18:55:22,657 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:22,657 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:22,657 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:22,657 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:22,657 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:22,657 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:23,036 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/9cc8be2569144b7f8f627c4dfd4587dc 2023-03-08 18:55:23,041 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/00a3dc9d388b42da8cd5f72975734740 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/00a3dc9d388b42da8cd5f72975734740 2023-03-08 18:55:23,045 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/00a3dc9d388b42da8cd5f72975734740, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 18:55:23,045 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/9cc8be2569144b7f8f627c4dfd4587dc as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/9cc8be2569144b7f8f627c4dfd4587dc 2023-03-08 18:55:23,049 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/9cc8be2569144b7f8f627c4dfd4587dc, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:55:23,050 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 924cd1dc6b0366a92e4bc7c5a4762a13 in 1270ms, sequenceid=9, compaction requested=false 2023-03-08 18:55:23,050 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:23,050 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:23,050 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:23,050 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:23,050 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,050 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 18:55:23,050 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 18:55:23,051 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,052 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:23,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:23,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:23,052 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:23,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:23,052 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:23,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:23,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:23,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:23,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 18:55:23,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a57af2f[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:23,053 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 18:55:23,053 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:23,056 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:23,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:23,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:23,056 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,056 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 18:55:23,056 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 18:55:23,056 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:23,056 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:23,057 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 18:55:23,057 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,057 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:23,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:23,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:23,057 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 18:55:23,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:23,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:23,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:23,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:23,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:23,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:23,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 18:55:23,059 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 18:55:23,059 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:23,059 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:23,059 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 18:55:23,059 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:23,061 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,061 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:23,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:23,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,061 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:23,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:23,061 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:23,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,063 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:23,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:23,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:23,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 18:55:23,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:23,065 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:23,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:23,065 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,080 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:55:23,165 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:23,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:23,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:23,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:23,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:23,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:23,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:23,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 18:55:23,173 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 18:55:23,173 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:23,173 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 18:55:23,173 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 18:55:23,173 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,173 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:23,175 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:23,175 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:23,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:23,176 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:23,176 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:23,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:23,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:23,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:23,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:23,177 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 18:55:23,178 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 18:55:23,178 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 18:55:23,178 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 18:55:23,178 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:23,179 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 18:55:23,179 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:23,179 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:23,179 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. started... 2023-03-08 18:55:23,179 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:23,179 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:23,179 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 70855e40ae020380397c6a9b2ff4c892 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:55:23,179 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. started... 2023-03-08 18:55:23,179 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. started... 2023-03-08 18:55:23,179 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:23,179 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 087e27baac917de6bad03c6444a74b6d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:55:23,179 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 924cd1dc6b0366a92e4bc7c5a4762a13 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:55:23,595 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992 2023-03-08 18:55:23,597 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1 2023-03-08 18:55:23,598 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18 2023-03-08 18:55:23,605 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/5d0f81d8cf7f4498add583a785c93576 2023-03-08 18:55:23,605 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/bd3164ac449643289a677b6bf46c83f9 2023-03-08 18:55:24,003 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/49bf9da251214ae7b2178764443f2a22 2023-03-08 18:55:24,018 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308f3781c97f10647609f9185d4cbea4ddf to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f3781c97f10647609f9185d4cbea4ddf 2023-03-08 18:55:24,019 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b0face12dca24f4194e047ea68763dea to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b0face12dca24f4194e047ea68763dea 2023-03-08 18:55:24,023 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/4837d54c0e78408ab1fa203e87fa07e9 2023-03-08 18:55:24,027 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/bd3164ac449643289a677b6bf46c83f9 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/bd3164ac449643289a677b6bf46c83f9 2023-03-08 18:55:24,030 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/bd3164ac449643289a677b6bf46c83f9, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 18:55:24,030 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/4837d54c0e78408ab1fa203e87fa07e9 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/4837d54c0e78408ab1fa203e87fa07e9 2023-03-08 18:55:24,033 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/4837d54c0e78408ab1fa203e87fa07e9, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:55:24,034 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 70855e40ae020380397c6a9b2ff4c892 in 855ms, sequenceid=13, compaction requested=true 2023-03-08 18:55:24,034 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:24,034 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:24,034 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:24,417 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308a91c7d8060a94a55b65082c5d178c225 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308a91c7d8060a94a55b65082c5d178c225 2023-03-08 18:55:24,423 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/d39edff1a8db45eb9fb524f0d121653d 2023-03-08 18:55:24,426 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/5d0f81d8cf7f4498add583a785c93576 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/5d0f81d8cf7f4498add583a785c93576 2023-03-08 18:55:24,430 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/5d0f81d8cf7f4498add583a785c93576, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 18:55:24,430 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/d39edff1a8db45eb9fb524f0d121653d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/d39edff1a8db45eb9fb524f0d121653d 2023-03-08 18:55:24,434 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/d39edff1a8db45eb9fb524f0d121653d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:55:24,435 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 087e27baac917de6bad03c6444a74b6d in 1256ms, sequenceid=13, compaction requested=true 2023-03-08 18:55:24,435 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:24,435 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:24,435 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:24,822 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/451c93d932004dc6a9e72401f6bc0ae6 2023-03-08 18:55:24,827 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/49bf9da251214ae7b2178764443f2a22 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/49bf9da251214ae7b2178764443f2a22 2023-03-08 18:55:24,831 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/49bf9da251214ae7b2178764443f2a22, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 18:55:24,832 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/451c93d932004dc6a9e72401f6bc0ae6 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/451c93d932004dc6a9e72401f6bc0ae6 2023-03-08 18:55:24,837 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/451c93d932004dc6a9e72401f6bc0ae6, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:55:24,838 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 924cd1dc6b0366a92e4bc7c5a4762a13 in 1659ms, sequenceid=13, compaction requested=true 2023-03-08 18:55:24,838 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:24,838 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:24,838 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:24,838 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:24,838 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,838 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 18:55:24,838 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 18:55:24,840 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,840 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:24,840 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:24,840 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:24,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:24,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:24,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 18:55:24,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e57a066[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:24,841 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 18:55:24,841 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:24,843 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:24,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:24,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:24,843 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 18:55:24,843 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,843 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:24,843 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 18:55:24,843 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:24,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,848 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 18:55:24,848 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:24,848 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 18:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:24,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 18:55:24,850 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 18:55:24,850 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:24,850 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:24,850 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 18:55:24,850 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:24,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:24,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,852 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:24,852 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,853 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:24,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:24,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:24,856 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:24,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:24,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 18:55:24,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:24,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:24,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:24,956 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:24,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:24,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:24,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:24,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:24,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:24,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:24,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 18:55:24,963 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 18:55:24,963 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:24,963 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 18:55:24,963 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 18:55:24,964 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,964 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:24,966 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,966 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:24,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:24,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:24,967 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:24,967 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:24,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:24,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 18:55:24,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 18:55:24,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 18:55:24,971 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 18:55:24,971 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:24,971 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 18:55:24,971 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:24,971 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. started... 2023-03-08 18:55:24,971 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 924cd1dc6b0366a92e4bc7c5a4762a13 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:55:24,972 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:24,972 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. started... 2023-03-08 18:55:24,972 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:24,972 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. started... 2023-03-08 18:55:24,972 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 70855e40ae020380397c6a9b2ff4c892 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:55:24,972 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:24,972 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:24,972 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 087e27baac917de6bad03c6444a74b6d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:55:24,999 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1 2023-03-08 18:55:25,011 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b 2023-03-08 18:55:25,397 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9 2023-03-08 18:55:25,401 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/11fe3fe861b94ba28c3c71d689725dd4 2023-03-08 18:55:25,409 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/1bf5917d1e08401eb5729ee7799742d6 2023-03-08 18:55:25,419 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/b83ef318305a4c50852898001a0735b8 2023-03-08 18:55:25,423 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303086d394787048847a5b7d46c02ec3d4a6d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303086d394787048847a5b7d46c02ec3d4a6d 2023-03-08 18:55:25,428 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/9fd316deb3434201a8533d6648b1eed9 2023-03-08 18:55:25,432 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/1bf5917d1e08401eb5729ee7799742d6 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1bf5917d1e08401eb5729ee7799742d6 2023-03-08 18:55:25,433 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303089d0bb408105e4552877fc4afaa50b0b3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089d0bb408105e4552877fc4afaa50b0b3 2023-03-08 18:55:25,435 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1bf5917d1e08401eb5729ee7799742d6, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 18:55:25,436 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/9fd316deb3434201a8533d6648b1eed9 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/9fd316deb3434201a8533d6648b1eed9 2023-03-08 18:55:25,437 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/bb018b416cee4b28b129da16edfbb5d3 2023-03-08 18:55:25,440 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/9fd316deb3434201a8533d6648b1eed9, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:55:25,440 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 087e27baac917de6bad03c6444a74b6d in 468ms, sequenceid=17, compaction requested=true 2023-03-08 18:55:25,440 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:25,440 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:25,440 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:25,440 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/b83ef318305a4c50852898001a0735b8 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/b83ef318305a4c50852898001a0735b8 2023-03-08 18:55:25,444 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/b83ef318305a4c50852898001a0735b8, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 18:55:25,444 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/bb018b416cee4b28b129da16edfbb5d3 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/bb018b416cee4b28b129da16edfbb5d3 2023-03-08 18:55:25,448 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/bb018b416cee4b28b129da16edfbb5d3, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:55:25,449 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 924cd1dc6b0366a92e4bc7c5a4762a13 in 477ms, sequenceid=17, compaction requested=true 2023-03-08 18:55:25,449 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:25,449 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:25,449 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:25,814 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030812812e618618431a86340e22ebe7c0b6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030812812e618618431a86340e22ebe7c0b6 2023-03-08 18:55:26,219 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/6caa146fb006423983892ce72dc16ee3 2023-03-08 18:55:26,223 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/11fe3fe861b94ba28c3c71d689725dd4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/11fe3fe861b94ba28c3c71d689725dd4 2023-03-08 18:55:26,227 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/11fe3fe861b94ba28c3c71d689725dd4, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 18:55:26,228 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/6caa146fb006423983892ce72dc16ee3 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/6caa146fb006423983892ce72dc16ee3 2023-03-08 18:55:26,232 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/6caa146fb006423983892ce72dc16ee3, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:55:26,232 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 70855e40ae020380397c6a9b2ff4c892 in 1260ms, sequenceid=17, compaction requested=true 2023-03-08 18:55:26,233 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:26,233 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:26,233 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:26,233 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:26,233 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,233 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 18:55:26,233 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 18:55:26,234 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,234 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:26,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:26,235 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:26,235 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 18:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@268a4a93[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:26,236 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 18:55:26,236 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:26,238 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:26,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:26,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:26,238 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 18:55:26,238 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 18:55:26,238 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:26,238 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,238 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:26,239 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 18:55:26,239 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:26,239 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,239 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 18:55:26,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:26,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:26,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:26,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:26,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:26,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:26,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:26,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:26,241 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 18:55:26,241 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:26,241 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:26,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 18:55:26,241 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 18:55:26,241 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:26,248 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:26,248 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:26,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:26,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:26,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:26,248 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:26,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:26,249 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:26,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:26,250 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:26,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:26,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,252 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,252 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:26,252 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:26,252 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,252 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:26,252 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:26,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 18:55:26,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:26,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:26,252 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:26,252 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:26,252 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:26,253 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:26,253 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:26,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:26,253 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:26,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:26,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:26,353 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:26,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:26,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1580): Client=jenkins//172.31.2.20 snapshot request for:{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 18:55:26,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678301726364 (current time:1678301726364). 2023-03-08 18:55:26,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-08 18:55:26,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] zookeeper.ReadOnlyZKClient(139): Connect 0x1b02d196 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:55:26,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@43d73cd8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:55:26,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:55:26,377 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:38896, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:55:26,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1b02d196 to 127.0.0.1:54098 2023-03-08 18:55:26,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:26,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-08 18:55:26,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 18:55:26,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 18:55:26,390 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 18:55:26,390 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678302326390 2023-03-08 18:55:26,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 18:55:26,391 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1678301726354 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-08 18:55:26,392 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 18:55:26,392 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 18:55:26,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 4 msec 2023-03-08 18:55:26,393 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 18:55:26,394 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1678302326394 2023-03-08 18:55:26,396 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:26,396 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-08 18:55:26,496 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:26,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:26,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:26,500 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-08 18:55:26,700 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:26,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:26,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:26,701 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-08 18:55:26,800 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1678301726354 2023-03-08 18:55:26,800 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1678301726354' 2023-03-08 18:55:26,800 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-08 18:55:26,800 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1678301726354' starting 'acquire' 2023-03-08 18:55:26,800 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1678301726354', kicking off acquire phase on members. 2023-03-08 18:55:26,801 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:26,801 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1678301726354 2023-03-08 18:55:26,802 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-08 18:55:26,802 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-08 18:55:26,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 18:55:26,803 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,803 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:26,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1678301726354 2023-03-08 18:55:26,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:26,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-08 18:55:26,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1678301726354 2023-03-08 18:55:26,803 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1678301726354 from table testCompactionWithHFileLink type FLUSH 2023-03-08 18:55:26,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1678301726354 2023-03-08 18:55:26,806 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1678301726354' with timeout 300000ms 2023-03-08 18:55:26,806 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-08 18:55:26,806 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1678301726354' starting 'acquire' stage 2023-03-08 18:55:26,806 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1678301726354' locally acquired 2023-03-08 18:55:26,806 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (snaptb-1678301726354) in zk 2023-03-08 18:55:26,808 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,808 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1678301726354 2023-03-08 18:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 18:55:26,808 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678301726354 2023-03-08 18:55:26,808 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1678301726354' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:26,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678301726354 2023-03-08 18:55:26,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:26,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'snaptb-1678301726354' on coordinator 2023-03-08 18:55:26,809 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1678301726354' starting 'in-barrier' execution. 2023-03-08 18:55:26,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55867302[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:26,809 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1678301726354 2023-03-08 18:55:26,811 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678301726354 2023-03-08 18:55:26,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1678301726354 2023-03-08 18:55:26,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1678301726354 2023-03-08 18:55:26,811 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1678301726354' received 'reached' from coordinator. 2023-03-08 18:55:26,811 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:26,811 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:26,812 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-08 18:55:26,812 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. started... 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. started... 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. started... 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:26,812 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:26,813 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.' region-info for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,813 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.' region-info for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,813 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.' region-info for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,980 INFO [regionserver/jenkins-hbase6:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-08 18:55:26,983 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 18:55:26,983 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 18:55:26,983 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 18:55:26,987 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/8e47fa9593bd4665acef00d8eb428147, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/5d0f81d8cf7f4498add583a785c93576, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1bf5917d1e08401eb5729ee7799742d6] hfiles 2023-03-08 18:55:26,987 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/a7594ec12b084addbcc1369ec9928778, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/00a3dc9d388b42da8cd5f72975734740, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/49bf9da251214ae7b2178764443f2a22, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/b83ef318305a4c50852898001a0735b8] hfiles 2023-03-08 18:55:26,987 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/1f9fdb228c8647fb87719f5cd2ad1bc0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/c8c661ff321b4588b2eb3fea2a3e6a20, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/bd3164ac449643289a677b6bf46c83f9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/11fe3fe861b94ba28c3c71d689725dd4] hfiles 2023-03-08 18:55:26,987 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/1f9fdb228c8647fb87719f5cd2ad1bc0 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,987 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/a7594ec12b084addbcc1369ec9928778 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,987 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/8e47fa9593bd4665acef00d8eb428147 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/c8c661ff321b4588b2eb3fea2a3e6a20 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/00a3dc9d388b42da8cd5f72975734740 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/49bf9da251214ae7b2178764443f2a22 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/bd3164ac449643289a677b6bf46c83f9 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/b83ef318305a4c50852898001a0735b8 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/5d0f81d8cf7f4498add583a785c93576 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/11fe3fe861b94ba28c3c71d689725dd4 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/cdcbd35b856741ea902048713afcc63a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/9cc8be2569144b7f8f627c4dfd4587dc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/451c93d932004dc6a9e72401f6bc0ae6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/bb018b416cee4b28b129da16edfbb5d3] hfiles 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/8cf2aef7d2a74556b2cb4eb00f828dba, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/2ea9681be9bb4ce8b6811c960e6d58c7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/4837d54c0e78408ab1fa203e87fa07e9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/6caa146fb006423983892ce72dc16ee3] hfiles 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/8cf2aef7d2a74556b2cb4eb00f828dba for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1bf5917d1e08401eb5729ee7799742d6 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/2ea9681be9bb4ce8b6811c960e6d58c7 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/3f86ffff34c0468fb5aa1b8d80d7fc87, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/a154990d75dd4a3681e935917dce1002, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/d39edff1a8db45eb9fb524f0d121653d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/9fd316deb3434201a8533d6648b1eed9] hfiles 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/4837d54c0e78408ab1fa203e87fa07e9 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/3f86ffff34c0468fb5aa1b8d80d7fc87 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/6caa146fb006423983892ce72dc16ee3 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/a154990d75dd4a3681e935917dce1002 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,990 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/cdcbd35b856741ea902048713afcc63a for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/d39edff1a8db45eb9fb524f0d121653d for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/9fd316deb3434201a8533d6648b1eed9 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/9cc8be2569144b7f8f627c4dfd4587dc for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/451c93d932004dc6a9e72401f6bc0ae6 for snapshot=snaptb-1678301726354 2023-03-08 18:55:26,991 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/bb018b416cee4b28b129da16edfbb5d3 for snapshot=snaptb-1678301726354 2023-03-08 18:55:27,001 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:27,003 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. completed. 2023-03-08 18:55:27,003 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:27,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:27,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:27,003 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-08 18:55:27,003 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-08 18:55:27,004 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. completed. 2023-03-08 18:55:27,004 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:27,004 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-08 18:55:27,402 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. completed. 2023-03-08 18:55:27,402 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:27,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-08 18:55:27,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-08 18:55:27,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1678301726354' locally completed 2023-03-08 18:55:27,402 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1678301726354' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:27,406 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1678301726354' has notified controller of completion 2023-03-08 18:55:27,406 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,406 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:27,406 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1678301726354' completed. 2023-03-08 18:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 18:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678301726354 2023-03-08 18:55:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678301726354 2023-03-08 18:55:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1678301726354' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:27,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'snaptb-1678301726354', counting down latch. Waiting for 0 more 2023-03-08 18:55:27,408 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1678301726354' execution completed 2023-03-08 18:55:27,408 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:27,408 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:27,408 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1678301726354 2023-03-08 18:55:27,408 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1678301726354including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-08 18:55:27,409 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:27,410 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:27,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:27,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:27,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 18:55:27,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:27,410 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-08 18:55:27,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:27,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:27,410 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-08 18:55:27,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 18:55:27,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678301726354 2023-03-08 18:55:27,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:27,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:27,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678301726354 2023-03-08 18:55:27,411 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:27,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678301726354 2023-03-08 18:55:27,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,413 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-08 18:55:27,413 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,413 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:27,414 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1678301726354 2023-03-08 18:55:27,413 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678301726354 2023-03-08 18:55:27,414 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-08 18:55:27,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-08 18:55:27,414 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1678301726354 2023-03-08 18:55:27,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 18:55:27,413 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-08 18:55:27,414 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-08 18:55:27,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1678301726354 because node does not exist (not an error) 2023-03-08 18:55:27,414 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678301726354 2023-03-08 18:55:27,414 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678301726354/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:27,414 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678301726354 2023-03-08 18:55:27,414 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678301726354 2023-03-08 18:55:27,414 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678301726354 2023-03-08 18:55:27,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-08 18:55:27,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f] mob files 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18 2023-03-08 18:55:27,415 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303087d11bc475e9e40cb8c8ce3a6dd44bc19, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089b7f1f4647b2476694cb6c5718ae3116, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089d0bb408105e4552877fc4afaa50b0b3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308a91c7d8060a94a55b65082c5d178c225, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030850c9ef639bbc43b4abcec48683ce6184, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303086d394787048847a5b7d46c02ec3d4a6d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303089a800a8a78da4ab5b1a367f95f5f08a7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f3781c97f10647609f9185d4cbea4ddf, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030812812e618618431a86340e22ebe7c0b6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b0face12dca24f4194e047ea68763dea, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d79cb2624513481c98336e55882b921d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308daa3430e10524dec9d5fa13a1effa3da] mob files 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303087d11bc475e9e40cb8c8ce3a6dd44bc19 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089b7f1f4647b2476694cb6c5718ae3116 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089d0bb408105e4552877fc4afaa50b0b3 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308a91c7d8060a94a55b65082c5d178c225 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030850c9ef639bbc43b4abcec48683ce6184 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303086d394787048847a5b7d46c02ec3d4a6d 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303089a800a8a78da4ab5b1a367f95f5f08a7 2023-03-08 18:55:27,416 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f3781c97f10647609f9185d4cbea4ddf 2023-03-08 18:55:27,417 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030812812e618618431a86340e22ebe7c0b6 2023-03-08 18:55:27,417 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b0face12dca24f4194e047ea68763dea 2023-03-08 18:55:27,417 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d79cb2624513481c98336e55882b921d 2023-03-08 18:55:27,417 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308daa3430e10524dec9d5fa13a1effa3da 2023-03-08 18:55:27,503 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:27,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:27,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:27,505 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-08 18:55:27,823 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1678301726354 2023-03-08 18:55:27,825 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp/snaptb-1678301726354 2023-03-08 18:55:28,272 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp/snaptb-1678301726354 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/snaptb-1678301726354 2023-03-08 18:55:28,505 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:28,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:28,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:28,506 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-03-08 18:55:29,090 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1678301726354 of table testCompactionWithHFileLink completed 2023-03-08 18:55:29,090 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp/snaptb-1678301726354 2023-03-08 18:55:29,090 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp/snaptb-1678301726354 2023-03-08 18:55:29,091 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1678301726354 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678301726391Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1678301727823 (+1432 ms)Consolidate snapshot: snaptb-1678301726354 at 1678301727823Loading Region manifests for snaptb-1678301726354 at 1678301727823Writing data manifest for snaptb-1678301726354 at 1678301727832 (+9 ms)Verifying snapshot: snaptb-1678301726354 at 1678301728239 (+407 ms)Snapshot snaptb-1678301726354 of table testCompactionWithHFileLink completed at 1678301729090 (+851 ms) 2023-03-08 18:55:29,092 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 18:55:29,093 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.7000 sec 2023-03-08 18:55:30,506 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:30,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:30,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-08 18:55:30,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:30,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:30,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 18:55:30,541 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 18:55:30,541 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:30,541 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 18:55:30,541 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 18:55:30,542 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:30,542 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:30,543 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:30,543 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:30,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:30,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:30,544 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:30,544 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:30,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:30,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:30,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:30,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:30,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 18:55:30,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 18:55:30,546 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 18:55:30,546 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:30,546 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 18:55:30,547 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:30,547 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. started... 2023-03-08 18:55:30,547 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 924cd1dc6b0366a92e4bc7c5a4762a13 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:55:30,549 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:30,549 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:30,549 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:30,549 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. started... 2023-03-08 18:55:30,549 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:30,549 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. started... 2023-03-08 18:55:30,549 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 70855e40ae020380397c6a9b2ff4c892 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:55:30,549 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 087e27baac917de6bad03c6444a74b6d 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:55:30,974 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/67934cad6f434f73b61cd4b6201ffa26 2023-03-08 18:55:30,981 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 67934cad6f434f73b61cd4b6201ffa26 2023-03-08 18:55:31,376 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/cfc58fa1ec5040a182cd904e5f9036d1 2023-03-08 18:55:31,379 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/630a8287239042b0a7e119cc43049f6a 2023-03-08 18:55:31,380 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cfc58fa1ec5040a182cd904e5f9036d1 2023-03-08 18:55:31,382 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 630a8287239042b0a7e119cc43049f6a 2023-03-08 18:55:31,391 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/bc235f6bde2f4b4a8b2bc48bafe3540f 2023-03-08 18:55:31,394 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc235f6bde2f4b4a8b2bc48bafe3540f 2023-03-08 18:55:31,395 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/67934cad6f434f73b61cd4b6201ffa26 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/67934cad6f434f73b61cd4b6201ffa26 2023-03-08 18:55:31,398 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 67934cad6f434f73b61cd4b6201ffa26 2023-03-08 18:55:31,398 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/67934cad6f434f73b61cd4b6201ffa26, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 18:55:31,398 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/bc235f6bde2f4b4a8b2bc48bafe3540f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/bc235f6bde2f4b4a8b2bc48bafe3540f 2023-03-08 18:55:31,402 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc235f6bde2f4b4a8b2bc48bafe3540f 2023-03-08 18:55:31,402 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/bc235f6bde2f4b4a8b2bc48bafe3540f, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 18:55:31,403 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 087e27baac917de6bad03c6444a74b6d in 854ms, sequenceid=24, compaction requested=true 2023-03-08 18:55:31,403 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:31,403 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:31,403 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:32,193 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/fb3ed379475d412da34e6b4794084fd0 2023-03-08 18:55:32,196 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/57a57eaaa7d44a6693a9d57cbc306a92 2023-03-08 18:55:32,197 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb3ed379475d412da34e6b4794084fd0 2023-03-08 18:55:32,198 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/cfc58fa1ec5040a182cd904e5f9036d1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/cfc58fa1ec5040a182cd904e5f9036d1 2023-03-08 18:55:32,199 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57a57eaaa7d44a6693a9d57cbc306a92 2023-03-08 18:55:32,200 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/630a8287239042b0a7e119cc43049f6a as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/630a8287239042b0a7e119cc43049f6a 2023-03-08 18:55:32,201 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cfc58fa1ec5040a182cd904e5f9036d1 2023-03-08 18:55:32,201 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/cfc58fa1ec5040a182cd904e5f9036d1, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 18:55:32,202 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/fb3ed379475d412da34e6b4794084fd0 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/fb3ed379475d412da34e6b4794084fd0 2023-03-08 18:55:32,204 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 630a8287239042b0a7e119cc43049f6a 2023-03-08 18:55:32,204 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/630a8287239042b0a7e119cc43049f6a, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 18:55:32,205 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/57a57eaaa7d44a6693a9d57cbc306a92 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/57a57eaaa7d44a6693a9d57cbc306a92 2023-03-08 18:55:32,205 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb3ed379475d412da34e6b4794084fd0 2023-03-08 18:55:32,206 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/fb3ed379475d412da34e6b4794084fd0, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 18:55:32,206 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 70855e40ae020380397c6a9b2ff4c892 in 1657ms, sequenceid=24, compaction requested=true 2023-03-08 18:55:32,206 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:32,206 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:32,206 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:32,208 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57a57eaaa7d44a6693a9d57cbc306a92 2023-03-08 18:55:32,208 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/57a57eaaa7d44a6693a9d57cbc306a92, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 18:55:32,209 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 924cd1dc6b0366a92e4bc7c5a4762a13 in 1662ms, sequenceid=24, compaction requested=true 2023-03-08 18:55:32,209 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:32,209 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:32,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:32,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:32,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 18:55:32,209 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 18:55:32,211 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,211 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:32,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:32,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:32,212 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:32,212 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:32,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:32,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:32,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:32,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:32,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 18:55:32,213 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 18:55:32,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@720f804a[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:32,213 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:32,217 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:32,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:32,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:32,217 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 18:55:32,217 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 18:55:32,217 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:32,217 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,217 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:32,219 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,219 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 18:55:32,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:32,219 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:32,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:32,219 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 18:55:32,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:32,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:32,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:32,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:32,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:32,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:32,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 18:55:32,221 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 18:55:32,221 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:32,221 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:32,221 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 18:55:32,221 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:32,223 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:32,223 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:32,223 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:32,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:32,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:32,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:32,223 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:32,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:32,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:32,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:32,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:32,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,225 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:32,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 18:55:32,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,227 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,227 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:32,227 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:32,227 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 18:55:32,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:32,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:32,227 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:32,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 18:55:32,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:32,227 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:32,227 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:32,228 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:32,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 18:55:32,228 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 18:55:32,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:32,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:32,228 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 18:55:32,228 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:32,228 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:32,328 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:32,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:32,330 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:32,330 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 924cd1dc6b0366a92e4bc7c5a4762a13/family1 is initiating major compaction (all files) 2023-03-08 18:55:32,330 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:32,330 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:32,331 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 924cd1dc6b0366a92e4bc7c5a4762a13/family1 in testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:32,331 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/a7594ec12b084addbcc1369ec9928778, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/00a3dc9d388b42da8cd5f72975734740, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/49bf9da251214ae7b2178764443f2a22, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/b83ef318305a4c50852898001a0735b8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/630a8287239042b0a7e119cc43049f6a] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp, totalSize=26.7 K 2023-03-08 18:55:32,331 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting a7594ec12b084addbcc1369ec9928778, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719975 2023-03-08 18:55:32,332 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 00a3dc9d388b42da8cd5f72975734740, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721767 2023-03-08 18:55:32,332 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 49bf9da251214ae7b2178764443f2a22, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723168 2023-03-08 18:55:32,332 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting b83ef318305a4c50852898001a0735b8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724959 2023-03-08 18:55:32,333 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 630a8287239042b0a7e119cc43049f6a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 18:55:32,345 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 924cd1dc6b0366a92e4bc7c5a4762a13#family1#compaction#108 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:32,753 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308c399da040ef14d30af17d249b012494f_del 2023-03-08 18:55:32,753 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308c399da040ef14d30af17d249b012494f_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c399da040ef14d30af17d249b012494f_del 2023-03-08 18:55:33,566 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for acbf263a42414626a4d08c722ccd6bd2 2023-03-08 18:55:33,567 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/acbf263a42414626a4d08c722ccd6bd2 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/acbf263a42414626a4d08c722ccd6bd2 2023-03-08 18:55:33,570 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for acbf263a42414626a4d08c722ccd6bd2 2023-03-08 18:55:33,571 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 924cd1dc6b0366a92e4bc7c5a4762a13/family1 of 924cd1dc6b0366a92e4bc7c5a4762a13 into acbf263a42414626a4d08c722ccd6bd2(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:33,571 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:33,571 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:33,571 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 924cd1dc6b0366a92e4bc7c5a4762a13/family2 is initiating major compaction (all files) 2023-03-08 18:55:33,571 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:33,571 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:33,572 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 924cd1dc6b0366a92e4bc7c5a4762a13/family2 in testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:33,572 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/cdcbd35b856741ea902048713afcc63a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/9cc8be2569144b7f8f627c4dfd4587dc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/451c93d932004dc6a9e72401f6bc0ae6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/bb018b416cee4b28b129da16edfbb5d3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/57a57eaaa7d44a6693a9d57cbc306a92] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp, totalSize=25.4 K 2023-03-08 18:55:33,572 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting cdcbd35b856741ea902048713afcc63a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719975 2023-03-08 18:55:33,573 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 9cc8be2569144b7f8f627c4dfd4587dc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721767 2023-03-08 18:55:33,573 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 451c93d932004dc6a9e72401f6bc0ae6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723168 2023-03-08 18:55:33,573 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting bb018b416cee4b28b129da16edfbb5d3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724959 2023-03-08 18:55:33,573 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 57a57eaaa7d44a6693a9d57cbc306a92, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 18:55:33,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/a7594ec12b084addbcc1369ec9928778, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/00a3dc9d388b42da8cd5f72975734740, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/49bf9da251214ae7b2178764443f2a22, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/b83ef318305a4c50852898001a0735b8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/630a8287239042b0a7e119cc43049f6a] to archive 2023-03-08 18:55:33,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:33,584 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/a7594ec12b084addbcc1369ec9928778 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/a7594ec12b084addbcc1369ec9928778 2023-03-08 18:55:33,584 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 924cd1dc6b0366a92e4bc7c5a4762a13#family2#compaction#109 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:33,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/00a3dc9d388b42da8cd5f72975734740 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/00a3dc9d388b42da8cd5f72975734740 2023-03-08 18:55:33,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/49bf9da251214ae7b2178764443f2a22 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/49bf9da251214ae7b2178764443f2a22 2023-03-08 18:55:33,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/b83ef318305a4c50852898001a0735b8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/b83ef318305a4c50852898001a0735b8 2023-03-08 18:55:33,588 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/630a8287239042b0a7e119cc43049f6a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/630a8287239042b0a7e119cc43049f6a 2023-03-08 18:55:33,993 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308d940c42fdffd42efb7652de862bbd7ac_del 2023-03-08 18:55:33,993 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d940c42fdffd42efb7652de862bbd7ac_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d940c42fdffd42efb7652de862bbd7ac_del 2023-03-08 18:55:34,807 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4fc3d409228b47fabd7b4d05105572d8 2023-03-08 18:55:34,807 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/4fc3d409228b47fabd7b4d05105572d8 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/4fc3d409228b47fabd7b4d05105572d8 2023-03-08 18:55:34,811 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4fc3d409228b47fabd7b4d05105572d8 2023-03-08 18:55:34,812 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 924cd1dc6b0366a92e4bc7c5a4762a13/family2 of 924cd1dc6b0366a92e4bc7c5a4762a13 into 4fc3d409228b47fabd7b4d05105572d8(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:34,812 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:34,812 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:34,813 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 70855e40ae020380397c6a9b2ff4c892/family1 is initiating major compaction (all files) 2023-03-08 18:55:34,813 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:34,813 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:34,814 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 70855e40ae020380397c6a9b2ff4c892/family1 in testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:34,814 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/1f9fdb228c8647fb87719f5cd2ad1bc0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/c8c661ff321b4588b2eb3fea2a3e6a20, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/bd3164ac449643289a677b6bf46c83f9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/11fe3fe861b94ba28c3c71d689725dd4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/cfc58fa1ec5040a182cd904e5f9036d1] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp, totalSize=26.7 K 2023-03-08 18:55:34,814 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 1f9fdb228c8647fb87719f5cd2ad1bc0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719972 2023-03-08 18:55:34,815 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting c8c661ff321b4588b2eb3fea2a3e6a20, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721766 2023-03-08 18:55:34,815 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting bd3164ac449643289a677b6bf46c83f9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723167 2023-03-08 18:55:34,815 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 11fe3fe861b94ba28c3c71d689725dd4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724958 2023-03-08 18:55:34,816 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting cfc58fa1ec5040a182cd904e5f9036d1, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 18:55:34,827 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 70855e40ae020380397c6a9b2ff4c892#family1#compaction#110 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:34,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/cdcbd35b856741ea902048713afcc63a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/9cc8be2569144b7f8f627c4dfd4587dc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/451c93d932004dc6a9e72401f6bc0ae6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/bb018b416cee4b28b129da16edfbb5d3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/57a57eaaa7d44a6693a9d57cbc306a92] to archive 2023-03-08 18:55:34,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:34,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/cdcbd35b856741ea902048713afcc63a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/cdcbd35b856741ea902048713afcc63a 2023-03-08 18:55:34,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/9cc8be2569144b7f8f627c4dfd4587dc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/9cc8be2569144b7f8f627c4dfd4587dc 2023-03-08 18:55:34,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/451c93d932004dc6a9e72401f6bc0ae6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/451c93d932004dc6a9e72401f6bc0ae6 2023-03-08 18:55:34,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/bb018b416cee4b28b129da16edfbb5d3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/bb018b416cee4b28b129da16edfbb5d3 2023-03-08 18:55:34,888 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/57a57eaaa7d44a6693a9d57cbc306a92 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/57a57eaaa7d44a6693a9d57cbc306a92 2023-03-08 18:55:35,241 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308db5e825bebfe44c58a43ecc15ecf7ff2_del 2023-03-08 18:55:35,241 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308db5e825bebfe44c58a43ecc15ecf7ff2_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308db5e825bebfe44c58a43ecc15ecf7ff2_del 2023-03-08 18:55:36,056 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48682ee375ec4b3fbf2e3f03032794d1 2023-03-08 18:55:36,057 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/48682ee375ec4b3fbf2e3f03032794d1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/48682ee375ec4b3fbf2e3f03032794d1 2023-03-08 18:55:36,061 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48682ee375ec4b3fbf2e3f03032794d1 2023-03-08 18:55:36,062 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 70855e40ae020380397c6a9b2ff4c892/family1 of 70855e40ae020380397c6a9b2ff4c892 into 48682ee375ec4b3fbf2e3f03032794d1(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:36,062 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:36,062 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:36,062 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 70855e40ae020380397c6a9b2ff4c892/family2 is initiating major compaction (all files) 2023-03-08 18:55:36,062 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:36,063 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:36,063 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 70855e40ae020380397c6a9b2ff4c892/family2 in testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:36,063 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/8cf2aef7d2a74556b2cb4eb00f828dba, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/2ea9681be9bb4ce8b6811c960e6d58c7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/4837d54c0e78408ab1fa203e87fa07e9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/6caa146fb006423983892ce72dc16ee3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/fb3ed379475d412da34e6b4794084fd0] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp, totalSize=25.4 K 2023-03-08 18:55:36,063 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 8cf2aef7d2a74556b2cb4eb00f828dba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719972 2023-03-08 18:55:36,064 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 2ea9681be9bb4ce8b6811c960e6d58c7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721766 2023-03-08 18:55:36,064 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 4837d54c0e78408ab1fa203e87fa07e9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723167 2023-03-08 18:55:36,064 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 6caa146fb006423983892ce72dc16ee3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724958 2023-03-08 18:55:36,065 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting fb3ed379475d412da34e6b4794084fd0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 18:55:36,076 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 70855e40ae020380397c6a9b2ff4c892#family2#compaction#111 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:36,082 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/1f9fdb228c8647fb87719f5cd2ad1bc0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/c8c661ff321b4588b2eb3fea2a3e6a20, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/bd3164ac449643289a677b6bf46c83f9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/11fe3fe861b94ba28c3c71d689725dd4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/cfc58fa1ec5040a182cd904e5f9036d1] to archive 2023-03-08 18:55:36,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:36,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/1f9fdb228c8647fb87719f5cd2ad1bc0 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/1f9fdb228c8647fb87719f5cd2ad1bc0 2023-03-08 18:55:36,086 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/c8c661ff321b4588b2eb3fea2a3e6a20 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/c8c661ff321b4588b2eb3fea2a3e6a20 2023-03-08 18:55:36,087 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/bd3164ac449643289a677b6bf46c83f9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/bd3164ac449643289a677b6bf46c83f9 2023-03-08 18:55:36,088 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/11fe3fe861b94ba28c3c71d689725dd4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/11fe3fe861b94ba28c3c71d689725dd4 2023-03-08 18:55:36,089 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/cfc58fa1ec5040a182cd904e5f9036d1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/cfc58fa1ec5040a182cd904e5f9036d1 2023-03-08 18:55:36,484 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303082f0e3eb1b49e49cca8eb1a732f521a55_del 2023-03-08 18:55:36,484 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303082f0e3eb1b49e49cca8eb1a732f521a55_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303082f0e3eb1b49e49cca8eb1a732f521a55_del 2023-03-08 18:55:37,298 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f4d17633ef3d41f286d746de62e88347 2023-03-08 18:55:37,299 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/f4d17633ef3d41f286d746de62e88347 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/f4d17633ef3d41f286d746de62e88347 2023-03-08 18:55:37,302 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f4d17633ef3d41f286d746de62e88347 2023-03-08 18:55:37,303 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 70855e40ae020380397c6a9b2ff4c892/family2 of 70855e40ae020380397c6a9b2ff4c892 into f4d17633ef3d41f286d746de62e88347(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:37,303 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:37,303 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:37,303 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 087e27baac917de6bad03c6444a74b6d/family1 is initiating major compaction (all files) 2023-03-08 18:55:37,303 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:37,303 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:37,303 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 087e27baac917de6bad03c6444a74b6d/family1 in testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:37,303 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/8e47fa9593bd4665acef00d8eb428147, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/5d0f81d8cf7f4498add583a785c93576, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1bf5917d1e08401eb5729ee7799742d6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/67934cad6f434f73b61cd4b6201ffa26] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp, totalSize=26.7 K 2023-03-08 18:55:37,304 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 8e47fa9593bd4665acef00d8eb428147, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719978 2023-03-08 18:55:37,304 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting fe3a899f5d434fb7b3863d0a7fe8cfb8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721767 2023-03-08 18:55:37,304 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 5d0f81d8cf7f4498add583a785c93576, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723168 2023-03-08 18:55:37,305 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 1bf5917d1e08401eb5729ee7799742d6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724959 2023-03-08 18:55:37,305 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 67934cad6f434f73b61cd4b6201ffa26, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 18:55:37,320 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 087e27baac917de6bad03c6444a74b6d#family1#compaction#112 average throughput is 0.24 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:37,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/8cf2aef7d2a74556b2cb4eb00f828dba, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/2ea9681be9bb4ce8b6811c960e6d58c7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/4837d54c0e78408ab1fa203e87fa07e9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/6caa146fb006423983892ce72dc16ee3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/fb3ed379475d412da34e6b4794084fd0] to archive 2023-03-08 18:55:37,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:37,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/8cf2aef7d2a74556b2cb4eb00f828dba to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/8cf2aef7d2a74556b2cb4eb00f828dba 2023-03-08 18:55:37,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/2ea9681be9bb4ce8b6811c960e6d58c7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/2ea9681be9bb4ce8b6811c960e6d58c7 2023-03-08 18:55:37,387 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/4837d54c0e78408ab1fa203e87fa07e9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/4837d54c0e78408ab1fa203e87fa07e9 2023-03-08 18:55:37,388 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/6caa146fb006423983892ce72dc16ee3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/6caa146fb006423983892ce72dc16ee3 2023-03-08 18:55:37,389 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/fb3ed379475d412da34e6b4794084fd0 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/fb3ed379475d412da34e6b4794084fd0 2023-03-08 18:55:37,730 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308aa2702d5a9994d16b9e72d884f02ee9d_del 2023-03-08 18:55:37,730 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308aa2702d5a9994d16b9e72d884f02ee9d_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308aa2702d5a9994d16b9e72d884f02ee9d_del 2023-03-08 18:55:38,543 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e3eb61c87354db8852fa2067847a0b1 2023-03-08 18:55:38,544 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/1e3eb61c87354db8852fa2067847a0b1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1e3eb61c87354db8852fa2067847a0b1 2023-03-08 18:55:38,547 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e3eb61c87354db8852fa2067847a0b1 2023-03-08 18:55:38,548 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 087e27baac917de6bad03c6444a74b6d/family1 of 087e27baac917de6bad03c6444a74b6d into 1e3eb61c87354db8852fa2067847a0b1(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:38,548 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:38,549 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:38,549 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 087e27baac917de6bad03c6444a74b6d/family2 is initiating major compaction (all files) 2023-03-08 18:55:38,549 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:38,549 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:38,549 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 087e27baac917de6bad03c6444a74b6d/family2 in testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:38,549 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/3f86ffff34c0468fb5aa1b8d80d7fc87, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/a154990d75dd4a3681e935917dce1002, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/d39edff1a8db45eb9fb524f0d121653d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/9fd316deb3434201a8533d6648b1eed9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/bc235f6bde2f4b4a8b2bc48bafe3540f] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp, totalSize=25.4 K 2023-03-08 18:55:38,549 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 3f86ffff34c0468fb5aa1b8d80d7fc87, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719978 2023-03-08 18:55:38,550 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting a154990d75dd4a3681e935917dce1002, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721767 2023-03-08 18:55:38,550 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting d39edff1a8db45eb9fb524f0d121653d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723168 2023-03-08 18:55:38,550 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 9fd316deb3434201a8533d6648b1eed9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724959 2023-03-08 18:55:38,551 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting bc235f6bde2f4b4a8b2bc48bafe3540f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 18:55:38,560 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 087e27baac917de6bad03c6444a74b6d#family2#compaction#113 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:38,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/8e47fa9593bd4665acef00d8eb428147, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/5d0f81d8cf7f4498add583a785c93576, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1bf5917d1e08401eb5729ee7799742d6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/67934cad6f434f73b61cd4b6201ffa26] to archive 2023-03-08 18:55:38,584 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:38,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/8e47fa9593bd4665acef00d8eb428147 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/8e47fa9593bd4665acef00d8eb428147 2023-03-08 18:55:38,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/fe3a899f5d434fb7b3863d0a7fe8cfb8 2023-03-08 18:55:38,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/5d0f81d8cf7f4498add583a785c93576 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/5d0f81d8cf7f4498add583a785c93576 2023-03-08 18:55:38,588 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1bf5917d1e08401eb5729ee7799742d6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1bf5917d1e08401eb5729ee7799742d6 2023-03-08 18:55:38,589 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/67934cad6f434f73b61cd4b6201ffa26 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/67934cad6f434f73b61cd4b6201ffa26 2023-03-08 18:55:38,967 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303082ed51fa1696645c7be25a9c59327ac3f_del 2023-03-08 18:55:38,968 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303082ed51fa1696645c7be25a9c59327ac3f_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303082ed51fa1696645c7be25a9c59327ac3f_del 2023-03-08 18:55:39,781 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8780c1ec59134344a47ae1be2a43d1f6 2023-03-08 18:55:39,782 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/8780c1ec59134344a47ae1be2a43d1f6 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/8780c1ec59134344a47ae1be2a43d1f6 2023-03-08 18:55:39,786 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8780c1ec59134344a47ae1be2a43d1f6 2023-03-08 18:55:39,787 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 087e27baac917de6bad03c6444a74b6d/family2 of 087e27baac917de6bad03c6444a74b6d into 8780c1ec59134344a47ae1be2a43d1f6(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:39,787 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:39,787 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/3f86ffff34c0468fb5aa1b8d80d7fc87, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/a154990d75dd4a3681e935917dce1002, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/d39edff1a8db45eb9fb524f0d121653d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/9fd316deb3434201a8533d6648b1eed9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/bc235f6bde2f4b4a8b2bc48bafe3540f] to archive 2023-03-08 18:55:39,788 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:39,791 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/3f86ffff34c0468fb5aa1b8d80d7fc87 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/3f86ffff34c0468fb5aa1b8d80d7fc87 2023-03-08 18:55:39,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/a154990d75dd4a3681e935917dce1002 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/a154990d75dd4a3681e935917dce1002 2023-03-08 18:55:39,792 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:39,793 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/d39edff1a8db45eb9fb524f0d121653d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/d39edff1a8db45eb9fb524f0d121653d 2023-03-08 18:55:39,795 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/9fd316deb3434201a8533d6648b1eed9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/9fd316deb3434201a8533d6648b1eed9 2023-03-08 18:55:39,796 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/bc235f6bde2f4b4a8b2bc48bafe3540f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/bc235f6bde2f4b4a8b2bc48bafe3540f 2023-03-08 18:55:39,797 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:39,802 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:39,803 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:39,812 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 18:55:39,842 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-08 18:55:39,845 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308db5e825bebfe44c58a43ecc15ecf7ff2_del 2023-03-08 18:55:39,848 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308c399da040ef14d30af17d249b012494f_del 2023-03-08 18:55:39,851 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308aa2702d5a9994d16b9e72d884f02ee9d_del 2023-03-08 18:55:39,851 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:39,851 DEBUG [Listener at localhost/36281] zookeeper.ReadOnlyZKClient(139): Connect 0x155ddbd8 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:55:39,856 DEBUG [Listener at localhost/36281] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18639285, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:55:39,856 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:39,856 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:39,857 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:40,328 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030888c2cbd947f743bcaffb3007e8967df6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030888c2cbd947f743bcaffb3007e8967df6 2023-03-08 18:55:40,329 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:40,348 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:55:40,352 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:42144, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:55:40,359 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:55:40,361 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:42680, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:55:40,369 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/63e843b018204d66994e74fd8bac3d11 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-08 18:55:40,370 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/63e843b018204d66994e74fd8bac3d11}] 2023-03-08 18:55:40,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/63e843b018204d66994e74fd8bac3d11 for inclusion in 70855e40ae020380397c6a9b2ff4c892/family1 2023-03-08 18:55:40,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-08 18:55:40,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 18:55:40,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:40,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/63e843b018204d66994e74fd8bac3d11 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__6td9if0bpotqvbn9doq84uoihpkpfs8fqm1tg12jc028n9i7sgf0gcfhmhu2hbol/family1/63e843b018204d66994e74fd8bac3d11 2023-03-08 18:55:40,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__6td9if0bpotqvbn9doq84uoihpkpfs8fqm1tg12jc028n9i7sgf0gcfhmhu2hbol/family1/63e843b018204d66994e74fd8bac3d11 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/179f63e361c54ab89dc33d84a20f991b_SeqId_29_ 2023-03-08 18:55:40,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__6td9if0bpotqvbn9doq84uoihpkpfs8fqm1tg12jc028n9i7sgf0gcfhmhu2hbol/family1/63e843b018204d66994e74fd8bac3d11 into 70855e40ae020380397c6a9b2ff4c892/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/179f63e361c54ab89dc33d84a20f991b_SeqId_29_ - updating store file list. 2023-03-08 18:55:40,381 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/179f63e361c54ab89dc33d84a20f991b_SeqId_29_ into 70855e40ae020380397c6a9b2ff4c892/family1 2023-03-08 18:55:40,381 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__6td9if0bpotqvbn9doq84uoihpkpfs8fqm1tg12jc028n9i7sgf0gcfhmhu2hbol/family1/63e843b018204d66994e74fd8bac3d11 into 70855e40ae020380397c6a9b2ff4c892/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/179f63e361c54ab89dc33d84a20f991b_SeqId_29_) 2023-03-08 18:55:40,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__6td9if0bpotqvbn9doq84uoihpkpfs8fqm1tg12jc028n9i7sgf0gcfhmhu2hbol/family1/63e843b018204d66994e74fd8bac3d11 2023-03-08 18:55:40,384 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:40,384 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:40,386 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9 2023-03-08 18:55:40,387 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361 2023-03-08 18:55:40,389 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18 2023-03-08 18:55:40,390 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f 2023-03-08 18:55:40,390 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 18:55:40,390 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 18:55:40,391 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:40,726 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308745b89b4c6e9498c93642d071aab4072 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308745b89b4c6e9498c93642d071aab4072 2023-03-08 18:55:40,726 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030820211c9f299b416b9b2cfc63c2c0f214 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030820211c9f299b416b9b2cfc63c2c0f214 2023-03-08 18:55:40,727 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:40,727 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:40,750 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/8fbc637267a943fb9c75649f0a7efe1f first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-08 18:55:40,752 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/8fbc637267a943fb9c75649f0a7efe1f}] 2023-03-08 18:55:40,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/8fbc637267a943fb9c75649f0a7efe1f for inclusion in 924cd1dc6b0366a92e4bc7c5a4762a13/family1 2023-03-08 18:55:40,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-08 18:55:40,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 18:55:40,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:40,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/8fbc637267a943fb9c75649f0a7efe1f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__pb7c8ljf6b2pd5pqp91vnfvmh60aovq8rc507363k897e88johgs3u4oj9dr386j/family1/8fbc637267a943fb9c75649f0a7efe1f 2023-03-08 18:55:40,760 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/3738dc6f84c44cc5b01ae738b17924bc first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-08 18:55:40,761 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/3738dc6f84c44cc5b01ae738b17924bc}] 2023-03-08 18:55:40,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__pb7c8ljf6b2pd5pqp91vnfvmh60aovq8rc507363k897e88johgs3u4oj9dr386j/family1/8fbc637267a943fb9c75649f0a7efe1f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/fd0d2c527dc04d8e98509c3adc056308_SeqId_29_ 2023-03-08 18:55:40,763 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/3738dc6f84c44cc5b01ae738b17924bc for inclusion in 087e27baac917de6bad03c6444a74b6d/family1 2023-03-08 18:55:40,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__pb7c8ljf6b2pd5pqp91vnfvmh60aovq8rc507363k897e88johgs3u4oj9dr386j/family1/8fbc637267a943fb9c75649f0a7efe1f into 924cd1dc6b0366a92e4bc7c5a4762a13/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/fd0d2c527dc04d8e98509c3adc056308_SeqId_29_ - updating store file list. 2023-03-08 18:55:40,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-08 18:55:40,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 18:55:40,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:40,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/3738dc6f84c44cc5b01ae738b17924bc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__akn06no96f1lspbmova5blr8815cd8ec170lag0g8qafa5r0q2stqnunn40l2n7v/family1/3738dc6f84c44cc5b01ae738b17924bc 2023-03-08 18:55:40,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/fd0d2c527dc04d8e98509c3adc056308_SeqId_29_ into 924cd1dc6b0366a92e4bc7c5a4762a13/family1 2023-03-08 18:55:40,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__pb7c8ljf6b2pd5pqp91vnfvmh60aovq8rc507363k897e88johgs3u4oj9dr386j/family1/8fbc637267a943fb9c75649f0a7efe1f into 924cd1dc6b0366a92e4bc7c5a4762a13/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/fd0d2c527dc04d8e98509c3adc056308_SeqId_29_) 2023-03-08 18:55:40,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__pb7c8ljf6b2pd5pqp91vnfvmh60aovq8rc507363k897e88johgs3u4oj9dr386j/family1/8fbc637267a943fb9c75649f0a7efe1f 2023-03-08 18:55:40,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__akn06no96f1lspbmova5blr8815cd8ec170lag0g8qafa5r0q2stqnunn40l2n7v/family1/3738dc6f84c44cc5b01ae738b17924bc as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/d71ba8a6b4584b8da830320b600d084f_SeqId_29_ 2023-03-08 18:55:40,770 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__akn06no96f1lspbmova5blr8815cd8ec170lag0g8qafa5r0q2stqnunn40l2n7v/family1/3738dc6f84c44cc5b01ae738b17924bc into 087e27baac917de6bad03c6444a74b6d/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/d71ba8a6b4584b8da830320b600d084f_SeqId_29_ - updating store file list. 2023-03-08 18:55:40,770 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:40,770 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:40,772 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b 2023-03-08 18:55:40,773 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/d71ba8a6b4584b8da830320b600d084f_SeqId_29_ into 087e27baac917de6bad03c6444a74b6d/family1 2023-03-08 18:55:40,773 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__akn06no96f1lspbmova5blr8815cd8ec170lag0g8qafa5r0q2stqnunn40l2n7v/family1/3738dc6f84c44cc5b01ae738b17924bc into 087e27baac917de6bad03c6444a74b6d/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/d71ba8a6b4584b8da830320b600d084f_SeqId_29_) 2023-03-08 18:55:40,773 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612 2023-03-08 18:55:40,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__akn06no96f1lspbmova5blr8815cd8ec170lag0g8qafa5r0q2stqnunn40l2n7v/family1/3738dc6f84c44cc5b01ae738b17924bc 2023-03-08 18:55:40,775 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:40,776 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae 2023-03-08 18:55:40,777 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:40,777 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992 2023-03-08 18:55:40,778 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 18:55:40,778 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 18:55:40,778 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323 2023-03-08 18:55:40,778 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:40,779 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db 2023-03-08 18:55:40,780 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1 2023-03-08 18:55:40,781 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1 2023-03-08 18:55:40,781 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 18:55:40,781 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 18:55:40,781 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:40,781 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:40,781 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:40,781 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(367): [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308db5e825bebfe44c58a43ecc15ecf7ff2_del] 2023-03-08 18:55:40,782 DEBUG [Listener at localhost/36281] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:40,783 DEBUG [Listener at localhost/36281] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308db5e825bebfe44c58a43ecc15ecf7ff2_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308db5e825bebfe44c58a43ecc15ecf7ff2_del 2023-03-08 18:55:40,783 INFO [Listener at localhost/36281] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 18:55:40,783 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(367): [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c399da040ef14d30af17d249b012494f_del] 2023-03-08 18:55:40,784 DEBUG [Listener at localhost/36281] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:40,785 DEBUG [Listener at localhost/36281] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c399da040ef14d30af17d249b012494f_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c399da040ef14d30af17d249b012494f_del 2023-03-08 18:55:40,785 INFO [Listener at localhost/36281] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 18:55:40,785 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(367): [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308aa2702d5a9994d16b9e72d884f02ee9d_del] 2023-03-08 18:55:40,785 DEBUG [Listener at localhost/36281] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:40,786 DEBUG [Listener at localhost/36281] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308aa2702d5a9994d16b9e72d884f02ee9d_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308aa2702d5a9994d16b9e72d884f02ee9d_del 2023-03-08 18:55:40,786 INFO [Listener at localhost/36281] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 18:55:40,789 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:40,791 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:40,794 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:40,795 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:40,801 INFO [Listener at localhost/36281] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-08 18:55:40,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testCompactionWithHFileLink 2023-03-08 18:55:40,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-08 18:55:40,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 18:55:40,805 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301740805"}]},"ts":"1678301740805"} 2023-03-08 18:55:40,807 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-08 18:55:40,808 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-08 18:55:40,809 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, UNASSIGN}] 2023-03-08 18:55:40,811 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, UNASSIGN 2023-03-08 18:55:40,811 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, UNASSIGN 2023-03-08 18:55:40,811 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, UNASSIGN 2023-03-08 18:55:40,811 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=087e27baac917de6bad03c6444a74b6d, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:40,811 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=924cd1dc6b0366a92e4bc7c5a4762a13, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:40,811 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301740811"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301740811"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301740811"}]},"ts":"1678301740811"} 2023-03-08 18:55:40,811 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301740811"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301740811"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301740811"}]},"ts":"1678301740811"} 2023-03-08 18:55:40,812 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=70855e40ae020380397c6a9b2ff4c892, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:40,812 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301740812"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301740812"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301740812"}]},"ts":"1678301740812"} 2023-03-08 18:55:40,813 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure 087e27baac917de6bad03c6444a74b6d, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:40,813 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 924cd1dc6b0366a92e4bc7c5a4762a13, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:40,814 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure 70855e40ae020380397c6a9b2ff4c892, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:40,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 18:55:40,964 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:40,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 087e27baac917de6bad03c6444a74b6d, disabling compactions & flushes 2023-03-08 18:55:40,965 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:40,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:40,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. after waiting 0 ms 2023-03-08 18:55:40,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:40,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 18:55:40,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:40,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:40,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:40,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:40,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 924cd1dc6b0366a92e4bc7c5a4762a13, disabling compactions & flushes 2023-03-08 18:55:40,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:40,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:40,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. after waiting 0 ms 2023-03-08 18:55:40,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:40,972 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=087e27baac917de6bad03c6444a74b6d, regionState=CLOSED 2023-03-08 18:55:40,972 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301740972"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301740972"}]},"ts":"1678301740972"} 2023-03-08 18:55:40,975 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=91 2023-03-08 18:55:40,975 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure 087e27baac917de6bad03c6444a74b6d, server=jenkins-hbase6.apache.org,46397,1678301621627 in 160 msec 2023-03-08 18:55:40,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 18:55:40,976 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, UNASSIGN in 166 msec 2023-03-08 18:55:40,977 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:40,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:40,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:40,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(109): Close 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:40,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 70855e40ae020380397c6a9b2ff4c892, disabling compactions & flushes 2023-03-08 18:55:40,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:40,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:40,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. after waiting 0 ms 2023-03-08 18:55:40,978 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=924cd1dc6b0366a92e4bc7c5a4762a13, regionState=CLOSED 2023-03-08 18:55:40,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:40,979 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301740978"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301740978"}]},"ts":"1678301740978"} 2023-03-08 18:55:40,981 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-03-08 18:55:40,981 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure 924cd1dc6b0366a92e4bc7c5a4762a13, server=jenkins-hbase6.apache.org,46397,1678301621627 in 167 msec 2023-03-08 18:55:40,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 18:55:40,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:40,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:40,983 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, UNASSIGN in 172 msec 2023-03-08 18:55:40,984 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(147): Closed 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:40,984 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=70855e40ae020380397c6a9b2ff4c892, regionState=CLOSED 2023-03-08 18:55:40,985 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301740984"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301740984"}]},"ts":"1678301740984"} 2023-03-08 18:55:40,986 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=89 2023-03-08 18:55:40,987 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure 70855e40ae020380397c6a9b2ff4c892, server=jenkins-hbase6.apache.org,46397,1678301621627 in 171 msec 2023-03-08 18:55:40,987 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-03-08 18:55:40,987 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, UNASSIGN in 178 msec 2023-03-08 18:55:40,988 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301740988"}]},"ts":"1678301740988"} 2023-03-08 18:55:40,989 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-08 18:55:40,990 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-08 18:55:40,992 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 189 msec 2023-03-08 18:55:41,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 18:55:41,107 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-08 18:55:41,117 INFO [Listener at localhost/36281] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1678301726354-1678301741117 2023-03-08 18:55:41,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1580): Client=jenkins//172.31.2.20 snapshot request for:{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 18:55:41,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678301741118 (current time:1678301741118). 2023-03-08 18:55:41,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-08 18:55:41,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] zookeeper.ReadOnlyZKClient(139): Connect 0x55a10372 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:55:41,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b1aedd7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:55:41,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:55:41,126 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:42148, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:55:41,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55a10372 to 127.0.0.1:54098 2023-03-08 18:55:41,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:55:41,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-08 18:55:41,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 18:55:41,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 18:55:41,129 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 18:55:41,130 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678302341130 2023-03-08 18:55:41,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 18:55:41,130 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1678301726354-1678301741117 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-08 18:55:41,130 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:41,130 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-08 18:55:41,231 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:41,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:41,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:41,232 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-08 18:55:41,432 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:41,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:41,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:41,433 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-08 18:55:41,538 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-08 18:55:41,540 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 18:55:41,540 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 18:55:41,540 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 18:55:41,540 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 18:55:41,540 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 18:55:41,540 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 18:55:41,540 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 18:55:41,540 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 18:55:41,543 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/179f63e361c54ab89dc33d84a20f991b_SeqId_29_, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/48682ee375ec4b3fbf2e3f03032794d1] hfiles 2023-03-08 18:55:41,543 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/179f63e361c54ab89dc33d84a20f991b_SeqId_29_ 2023-03-08 18:55:41,543 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/48682ee375ec4b3fbf2e3f03032794d1 2023-03-08 18:55:41,543 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1e3eb61c87354db8852fa2067847a0b1, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/d71ba8a6b4584b8da830320b600d084f_SeqId_29_] hfiles 2023-03-08 18:55:41,543 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/acbf263a42414626a4d08c722ccd6bd2, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/fd0d2c527dc04d8e98509c3adc056308_SeqId_29_] hfiles 2023-03-08 18:55:41,543 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1e3eb61c87354db8852fa2067847a0b1 2023-03-08 18:55:41,543 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/acbf263a42414626a4d08c722ccd6bd2 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/d71ba8a6b4584b8da830320b600d084f_SeqId_29_ 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/fd0d2c527dc04d8e98509c3adc056308_SeqId_29_ 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030820211c9f299b416b9b2cfc63c2c0f214, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308745b89b4c6e9498c93642d071aab4072, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030888c2cbd947f743bcaffb3007e8967df6] hfiles 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/f4d17633ef3d41f286d746de62e88347] hfiles 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030820211c9f299b416b9b2cfc63c2c0f214 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/f4d17633ef3d41f286d746de62e88347 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308745b89b4c6e9498c93642d071aab4072 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030888c2cbd947f743bcaffb3007e8967df6 2023-03-08 18:55:41,544 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/4fc3d409228b47fabd7b4d05105572d8] hfiles 2023-03-08 18:55:41,545 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/4fc3d409228b47fabd7b4d05105572d8 2023-03-08 18:55:41,545 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/8780c1ec59134344a47ae1be2a43d1f6] hfiles 2023-03-08 18:55:41,545 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/8780c1ec59134344a47ae1be2a43d1f6 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303087d11bc475e9e40cb8c8ce3a6dd44bc19, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089b7f1f4647b2476694cb6c5718ae3116, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089d0bb408105e4552877fc4afaa50b0b3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308a91c7d8060a94a55b65082c5d178c225, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d940c42fdffd42efb7652de862bbd7ac_del, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303082ed51fa1696645c7be25a9c59327ac3f_del, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030850c9ef639bbc43b4abcec48683ce6184, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303086d394787048847a5b7d46c02ec3d4a6d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303089a800a8a78da4ab5b1a367f95f5f08a7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f3781c97f10647609f9185d4cbea4ddf, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030812812e618618431a86340e22ebe7c0b6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303082f0e3eb1b49e49cca8eb1a732f521a55_del, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b0face12dca24f4194e047ea68763dea, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d79cb2624513481c98336e55882b921d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308daa3430e10524dec9d5fa13a1effa3da] hfiles 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303087d11bc475e9e40cb8c8ce3a6dd44bc19 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089b7f1f4647b2476694cb6c5718ae3116 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089d0bb408105e4552877fc4afaa50b0b3 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308a91c7d8060a94a55b65082c5d178c225 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d940c42fdffd42efb7652de862bbd7ac_del 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303082ed51fa1696645c7be25a9c59327ac3f_del 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030850c9ef639bbc43b4abcec48683ce6184 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303086d394787048847a5b7d46c02ec3d4a6d 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303089a800a8a78da4ab5b1a367f95f5f08a7 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f3781c97f10647609f9185d4cbea4ddf 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030812812e618618431a86340e22ebe7c0b6 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303082f0e3eb1b49e49cca8eb1a732f521a55_del 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308b0face12dca24f4194e047ea68763dea 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d79cb2624513481c98336e55882b921d 2023-03-08 18:55:41,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308daa3430e10524dec9d5fa13a1effa3da 2023-03-08 18:55:41,733 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:41,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:41,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:41,734 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-08 18:55:41,961 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-08 18:55:41,961 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1678301726354-1678301741117 2023-03-08 18:55:41,962 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678301726354-1678301741117 2023-03-08 18:55:42,235 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:42,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:42,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 18:55:42,236 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-08 18:55:42,392 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678301726354-1678301741117 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/hbase-failsafe-snaptb-1678301726354-1678301741117 2023-03-08 18:55:43,209 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1678301726354-1678301741117 of table testCompactionWithHFileLink completed 2023-03-08 18:55:43,209 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678301726354-1678301741117 2023-03-08 18:55:43,210 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678301726354-1678301741117 2023-03-08 18:55:43,210 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1678301726354-1678301741117 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678301741130Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1678301741538 (+408 ms)Consolidate snapshot: hbase-failsafe-snaptb-1678301726354-1678301741117 at 1678301741961 (+423 ms)Loading Region manifests for hbase-failsafe-snaptb-1678301726354-1678301741117 at 1678301741961Writing data manifest for hbase-failsafe-snaptb-1678301726354-1678301741117 at 1678301741965 (+4 ms)Verifying snapshot: hbase-failsafe-snaptb-1678301726354-1678301741117 at 1678301742377 (+412 ms)Snapshot hbase-failsafe-snaptb-1678301726354-1678301741117 of table testCompactionWithHFileLink completed at 1678301743209 (+832 ms) 2023-03-08 18:55:43,211 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 18:55:43,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0830 sec 2023-03-08 18:55:43,236 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 18:55:43,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 18:55:43,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1678301726354-1678301741117 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-08 18:55:43,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678301726354" table: "testCompactionWithHFileLink" creation_time: 1678301726364 type: FLUSH version: 2 ttl: 0 ) 2023-03-08 18:55:43,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1678301726354 as table=testCompactionWithHFileLink 2023-03-08 18:55:43,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 18:55:43,375 DEBUG [regionserver/jenkins-hbase6:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 18:55:43,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 18:55:43,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 18:55:43,693 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-08 18:55:43,694 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-08 18:55:43,694 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-08 18:55:43,695 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 18:55:43,700 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1678301726354" table: "testCompactionWithHFileLink" creation_time: 1678301726364 type: FLUSH version: 2 ttl: 0 2023-03-08 18:55:43,701 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink 2023-03-08 18:55:43,706 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-08 18:55:43,709 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030820211c9f299b416b9b2cfc63c2c0f214 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030820211c9f299b416b9b2cfc63c2c0f214 2023-03-08 18:55:43,711 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308745b89b4c6e9498c93642d071aab4072 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308745b89b4c6e9498c93642d071aab4072 2023-03-08 18:55:43,712 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030888c2cbd947f743bcaffb3007e8967df6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030888c2cbd947f743bcaffb3007e8967df6 2023-03-08 18:55:43,712 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,715 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612 of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,718 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,720 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992 of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,723 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323 of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,726 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,729 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1 of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,732 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1 of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,736 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9 of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,739 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361 of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,742 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18 of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,746 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f of snapshot=snaptb-1678301726354 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 18:55:43,751 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303082f0e3eb1b49e49cca8eb1a732f521a55_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303082f0e3eb1b49e49cca8eb1a732f521a55_del 2023-03-08 18:55:43,752 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d940c42fdffd42efb7652de862bbd7ac_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308d940c42fdffd42efb7652de862bbd7ac_del 2023-03-08 18:55:43,753 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303082ed51fa1696645c7be25a9c59327ac3f_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303082ed51fa1696645c7be25a9c59327ac3f_del 2023-03-08 18:55:43,753 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:43,753 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:43,753 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:43,756 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1e3eb61c87354db8852fa2067847a0b1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/1e3eb61c87354db8852fa2067847a0b1 2023-03-08 18:55:43,757 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/179f63e361c54ab89dc33d84a20f991b_SeqId_29_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/179f63e361c54ab89dc33d84a20f991b_SeqId_29_ 2023-03-08 18:55:43,757 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/acbf263a42414626a4d08c722ccd6bd2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/acbf263a42414626a4d08c722ccd6bd2 2023-03-08 18:55:43,758 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/d71ba8a6b4584b8da830320b600d084f_SeqId_29_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/d71ba8a6b4584b8da830320b600d084f_SeqId_29_ 2023-03-08 18:55:43,758 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8e47fa9593bd4665acef00d8eb428147 of snapshot=snaptb-1678301726354 to region=087e27baac917de6bad03c6444a74b6d table=testCompactionWithHFileLink 2023-03-08 18:55:43,758 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/48682ee375ec4b3fbf2e3f03032794d1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/48682ee375ec4b3fbf2e3f03032794d1 2023-03-08 18:55:43,758 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1f9fdb228c8647fb87719f5cd2ad1bc0 of snapshot=snaptb-1678301726354 to region=70855e40ae020380397c6a9b2ff4c892 table=testCompactionWithHFileLink 2023-03-08 18:55:43,758 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/fd0d2c527dc04d8e98509c3adc056308_SeqId_29_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/fd0d2c527dc04d8e98509c3adc056308_SeqId_29_ 2023-03-08 18:55:43,758 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a7594ec12b084addbcc1369ec9928778 of snapshot=snaptb-1678301726354 to region=924cd1dc6b0366a92e4bc7c5a4762a13 table=testCompactionWithHFileLink 2023-03-08 18:55:43,761 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fe3a899f5d434fb7b3863d0a7fe8cfb8 of snapshot=snaptb-1678301726354 to region=087e27baac917de6bad03c6444a74b6d table=testCompactionWithHFileLink 2023-03-08 18:55:43,762 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c8c661ff321b4588b2eb3fea2a3e6a20 of snapshot=snaptb-1678301726354 to region=70855e40ae020380397c6a9b2ff4c892 table=testCompactionWithHFileLink 2023-03-08 18:55:43,762 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 00a3dc9d388b42da8cd5f72975734740 of snapshot=snaptb-1678301726354 to region=924cd1dc6b0366a92e4bc7c5a4762a13 table=testCompactionWithHFileLink 2023-03-08 18:55:43,765 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5d0f81d8cf7f4498add583a785c93576 of snapshot=snaptb-1678301726354 to region=087e27baac917de6bad03c6444a74b6d table=testCompactionWithHFileLink 2023-03-08 18:55:43,765 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bd3164ac449643289a677b6bf46c83f9 of snapshot=snaptb-1678301726354 to region=70855e40ae020380397c6a9b2ff4c892 table=testCompactionWithHFileLink 2023-03-08 18:55:43,766 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 49bf9da251214ae7b2178764443f2a22 of snapshot=snaptb-1678301726354 to region=924cd1dc6b0366a92e4bc7c5a4762a13 table=testCompactionWithHFileLink 2023-03-08 18:55:43,769 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1bf5917d1e08401eb5729ee7799742d6 of snapshot=snaptb-1678301726354 to region=087e27baac917de6bad03c6444a74b6d table=testCompactionWithHFileLink 2023-03-08 18:55:43,769 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 11fe3fe861b94ba28c3c71d689725dd4 of snapshot=snaptb-1678301726354 to region=70855e40ae020380397c6a9b2ff4c892 table=testCompactionWithHFileLink 2023-03-08 18:55:43,769 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b83ef318305a4c50852898001a0735b8 of snapshot=snaptb-1678301726354 to region=924cd1dc6b0366a92e4bc7c5a4762a13 table=testCompactionWithHFileLink 2023-03-08 18:55:43,774 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/8780c1ec59134344a47ae1be2a43d1f6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/8780c1ec59134344a47ae1be2a43d1f6 2023-03-08 18:55:43,774 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/4fc3d409228b47fabd7b4d05105572d8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/4fc3d409228b47fabd7b4d05105572d8 2023-03-08 18:55:43,774 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3f86ffff34c0468fb5aa1b8d80d7fc87 of snapshot=snaptb-1678301726354 to region=087e27baac917de6bad03c6444a74b6d table=testCompactionWithHFileLink 2023-03-08 18:55:43,775 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cdcbd35b856741ea902048713afcc63a of snapshot=snaptb-1678301726354 to region=924cd1dc6b0366a92e4bc7c5a4762a13 table=testCompactionWithHFileLink 2023-03-08 18:55:43,774 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/f4d17633ef3d41f286d746de62e88347 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/f4d17633ef3d41f286d746de62e88347 2023-03-08 18:55:43,775 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8cf2aef7d2a74556b2cb4eb00f828dba of snapshot=snaptb-1678301726354 to region=70855e40ae020380397c6a9b2ff4c892 table=testCompactionWithHFileLink 2023-03-08 18:55:43,784 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a154990d75dd4a3681e935917dce1002 of snapshot=snaptb-1678301726354 to region=087e27baac917de6bad03c6444a74b6d table=testCompactionWithHFileLink 2023-03-08 18:55:43,784 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2ea9681be9bb4ce8b6811c960e6d58c7 of snapshot=snaptb-1678301726354 to region=70855e40ae020380397c6a9b2ff4c892 table=testCompactionWithHFileLink 2023-03-08 18:55:43,784 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9cc8be2569144b7f8f627c4dfd4587dc of snapshot=snaptb-1678301726354 to region=924cd1dc6b0366a92e4bc7c5a4762a13 table=testCompactionWithHFileLink 2023-03-08 18:55:43,788 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d39edff1a8db45eb9fb524f0d121653d of snapshot=snaptb-1678301726354 to region=087e27baac917de6bad03c6444a74b6d table=testCompactionWithHFileLink 2023-03-08 18:55:43,788 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4837d54c0e78408ab1fa203e87fa07e9 of snapshot=snaptb-1678301726354 to region=70855e40ae020380397c6a9b2ff4c892 table=testCompactionWithHFileLink 2023-03-08 18:55:43,788 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 451c93d932004dc6a9e72401f6bc0ae6 of snapshot=snaptb-1678301726354 to region=924cd1dc6b0366a92e4bc7c5a4762a13 table=testCompactionWithHFileLink 2023-03-08 18:55:43,791 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9fd316deb3434201a8533d6648b1eed9 of snapshot=snaptb-1678301726354 to region=087e27baac917de6bad03c6444a74b6d table=testCompactionWithHFileLink 2023-03-08 18:55:43,792 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bb018b416cee4b28b129da16edfbb5d3 of snapshot=snaptb-1678301726354 to region=924cd1dc6b0366a92e4bc7c5a4762a13 table=testCompactionWithHFileLink 2023-03-08 18:55:43,792 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6caa146fb006423983892ce72dc16ee3 of snapshot=snaptb-1678301726354 to region=70855e40ae020380397c6a9b2ff4c892 table=testCompactionWithHFileLink 2023-03-08 18:55:43,796 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1678301726354" table: "testCompactionWithHFileLink" creation_time: 1678301726364 type: FLUSH version: 2 ttl: 0 2023-03-08 18:55:43,797 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678301743797"}]},"ts":"9223372036854775807"} 2023-03-08 18:55:43,797 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678301743797"}]},"ts":"9223372036854775807"} 2023-03-08 18:55:43,797 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678301743797"}]},"ts":"9223372036854775807"} 2023-03-08 18:55:43,799 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-08 18:55:43,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 087e27baac917de6bad03c6444a74b6d, NAME => 'testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 70855e40ae020380397c6a9b2ff4c892, NAME => 'testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 924cd1dc6b0366a92e4bc7c5a4762a13, NAME => 'testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-08 18:55:43,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301743798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301743798"}]},"ts":"1678301743798"} 2023-03-08 18:55:43,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301743798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301743798"}]},"ts":"1678301743798"} 2023-03-08 18:55:43,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301743798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301743798"}]},"ts":"1678301743798"} 2023-03-08 18:55:43,800 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 18:55:43,800 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-08 18:55:43,800 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 087e27baac917de6bad03c6444a74b6d, NAME => 'testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 70855e40ae020380397c6a9b2ff4c892, NAME => 'testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 924cd1dc6b0366a92e4bc7c5a4762a13, NAME => 'testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-08 18:55:43,801 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1678301726354 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-08 18:55:43,803 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678301726354" table: "testCompactionWithHFileLink" creation_time: 1678301726364 type: FLUSH version: 2 ttl: 0 ) in 529 msec 2023-03-08 18:55:43,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 18:55:43,888 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-08 18:55:43,888 INFO [Listener at localhost/36281] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1678301726354-1678301741117 2023-03-08 18:55:43,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(748): Client=jenkins//172.31.2.20 delete name: "hbase-failsafe-snaptb-1678301726354-1678301741117" 2023-03-08 18:55:43,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1678301726354-1678301741117 2023-03-08 18:55:43,896 INFO [Listener at localhost/36281] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-08 18:55:43,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$10(2385): Client=jenkins//172.31.2.20 enable testCompactionWithHFileLink 2023-03-08 18:55:43,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-08 18:55:43,908 INFO [PEWorker-4] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-08 18:55:43,908 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301743908"}]},"ts":"1678301743908"} 2023-03-08 18:55:43,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 18:55:43,909 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-08 18:55:43,912 DEBUG [PEWorker-4] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-08 18:55:43,912 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, ASSIGN}] 2023-03-08 18:55:43,913 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, ASSIGN 2023-03-08 18:55:43,913 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, ASSIGN 2023-03-08 18:55:43,913 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, ASSIGN 2023-03-08 18:55:43,914 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 18:55:43,914 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 18:55:43,914 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 18:55:44,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 18:55:44,066 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=087e27baac917de6bad03c6444a74b6d, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:44,066 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=70855e40ae020380397c6a9b2ff4c892, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:44,066 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=924cd1dc6b0366a92e4bc7c5a4762a13, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:44,066 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301744066"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301744066"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301744066"}]},"ts":"1678301744066"} 2023-03-08 18:55:44,066 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301744066"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301744066"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301744066"}]},"ts":"1678301744066"} 2023-03-08 18:55:44,066 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301744066"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301744066"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301744066"}]},"ts":"1678301744066"} 2023-03-08 18:55:44,068 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 087e27baac917de6bad03c6444a74b6d, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:44,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 70855e40ae020380397c6a9b2ff4c892, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:44,069 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure 924cd1dc6b0366a92e4bc7c5a4762a13, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:44,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 18:55:44,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:44,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 70855e40ae020380397c6a9b2ff4c892, NAME => 'testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 18:55:44,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:44,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:44,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:44,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:44,223 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:44,224 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1 2023-03-08 18:55:44,224 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1 2023-03-08 18:55:44,225 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70855e40ae020380397c6a9b2ff4c892 columnFamilyName family1 2023-03-08 18:55:44,235 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-11fe3fe861b94ba28c3c71d689725dd4 2023-03-08 18:55:44,240 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-1f9fdb228c8647fb87719f5cd2ad1bc0 2023-03-08 18:55:44,244 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-bd3164ac449643289a677b6bf46c83f9 2023-03-08 18:55:44,249 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-c8c661ff321b4588b2eb3fea2a3e6a20 2023-03-08 18:55:44,249 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(310): Store=70855e40ae020380397c6a9b2ff4c892/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:44,249 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:44,250 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2 2023-03-08 18:55:44,250 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2 2023-03-08 18:55:44,251 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70855e40ae020380397c6a9b2ff4c892 columnFamilyName family2 2023-03-08 18:55:44,257 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-2ea9681be9bb4ce8b6811c960e6d58c7 2023-03-08 18:55:44,262 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-4837d54c0e78408ab1fa203e87fa07e9 2023-03-08 18:55:44,268 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-6caa146fb006423983892ce72dc16ee3 2023-03-08 18:55:44,273 DEBUG [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-8cf2aef7d2a74556b2cb4eb00f828dba 2023-03-08 18:55:44,273 INFO [StoreOpener-70855e40ae020380397c6a9b2ff4c892-1] regionserver.HStore(310): Store=70855e40ae020380397c6a9b2ff4c892/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:44,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:44,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:44,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:44,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:55:44,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 70855e40ae020380397c6a9b2ff4c892; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63265481, jitterRate=-0.05727086961269379}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:44,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:44,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892., pid=102, masterSystemTime=1678301744219 2023-03-08 18:55:44,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:55:44,279 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 18:55:44,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 18:55:44,279 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 18:55:44,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:44,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:44,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:44,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 924cd1dc6b0366a92e4bc7c5a4762a13, NAME => 'testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 18:55:44,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:44,281 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=70855e40ae020380397c6a9b2ff4c892, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:44,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:44,281 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301744281"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301744281"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301744281"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301744281"}]},"ts":"1678301744281"} 2023-03-08 18:55:44,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:44,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:44,283 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:44,283 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 18:55:44,283 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 18:55:44,284 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1912): 70855e40ae020380397c6a9b2ff4c892/family1 is initiating minor compaction (all files) 2023-03-08 18:55:44,284 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HStore(1912): 70855e40ae020380397c6a9b2ff4c892/family2 is initiating minor compaction (all files) 2023-03-08 18:55:44,284 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 70855e40ae020380397c6a9b2ff4c892/family1 in testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:44,284 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 70855e40ae020380397c6a9b2ff4c892/family2 in testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:55:44,284 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-1f9fdb228c8647fb87719f5cd2ad1bc0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-c8c661ff321b4588b2eb3fea2a3e6a20, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-bd3164ac449643289a677b6bf46c83f9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-11fe3fe861b94ba28c3c71d689725dd4] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp, totalSize=21.6 K 2023-03-08 18:55:44,284 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-8cf2aef7d2a74556b2cb4eb00f828dba, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-2ea9681be9bb4ce8b6811c960e6d58c7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-4837d54c0e78408ab1fa203e87fa07e9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-6caa146fb006423983892ce72dc16ee3] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp, totalSize=20.4 K 2023-03-08 18:55:44,284 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1 2023-03-08 18:55:44,284 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1 2023-03-08 18:55:44,284 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-03-08 18:55:44,284 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 70855e40ae020380397c6a9b2ff4c892, server=jenkins-hbase6.apache.org,46397,1678301621627 in 215 msec 2023-03-08 18:55:44,285 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 924cd1dc6b0366a92e4bc7c5a4762a13 columnFamilyName family1 2023-03-08 18:55:44,285 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=70855e40ae020380397c6a9b2ff4c892, ASSIGN in 372 msec 2023-03-08 18:55:44,285 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-8cf2aef7d2a74556b2cb4eb00f828dba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719972 2023-03-08 18:55:44,286 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-1f9fdb228c8647fb87719f5cd2ad1bc0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719972 2023-03-08 18:55:44,287 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-2ea9681be9bb4ce8b6811c960e6d58c7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721766 2023-03-08 18:55:44,287 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-c8c661ff321b4588b2eb3fea2a3e6a20, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721766 2023-03-08 18:55:44,288 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-4837d54c0e78408ab1fa203e87fa07e9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723167 2023-03-08 18:55:44,289 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-bd3164ac449643289a677b6bf46c83f9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723167 2023-03-08 18:55:44,289 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-6caa146fb006423983892ce72dc16ee3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724958 2023-03-08 18:55:44,295 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-11fe3fe861b94ba28c3c71d689725dd4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724958 2023-03-08 18:55:44,307 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-00a3dc9d388b42da8cd5f72975734740 2023-03-08 18:55:44,315 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-49bf9da251214ae7b2178764443f2a22 2023-03-08 18:55:44,322 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-a7594ec12b084addbcc1369ec9928778 2023-03-08 18:55:44,340 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 70855e40ae020380397c6a9b2ff4c892#family1#compaction#114 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:44,341 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] throttle.PressureAwareThroughputController(145): 70855e40ae020380397c6a9b2ff4c892#family2#compaction#115 average throughput is 0.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:44,341 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-b83ef318305a4c50852898001a0735b8 2023-03-08 18:55:44,341 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(310): Store=924cd1dc6b0366a92e4bc7c5a4762a13/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:44,342 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:44,344 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2 2023-03-08 18:55:44,344 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2 2023-03-08 18:55:44,345 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 924cd1dc6b0366a92e4bc7c5a4762a13 columnFamilyName family2 2023-03-08 18:55:44,361 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-451c93d932004dc6a9e72401f6bc0ae6 2023-03-08 18:55:44,368 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-9cc8be2569144b7f8f627c4dfd4587dc 2023-03-08 18:55:44,372 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-bb018b416cee4b28b129da16edfbb5d3 2023-03-08 18:55:44,376 DEBUG [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-cdcbd35b856741ea902048713afcc63a 2023-03-08 18:55:44,376 INFO [StoreOpener-924cd1dc6b0366a92e4bc7c5a4762a13-1] regionserver.HStore(310): Store=924cd1dc6b0366a92e4bc7c5a4762a13/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:44,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:44,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:44,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:44,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:55:44,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 924cd1dc6b0366a92e4bc7c5a4762a13; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63738157, jitterRate=-0.05022744834423065}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:44,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:44,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13., pid=103, masterSystemTime=1678301744219 2023-03-08 18:55:44,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 18:55:44,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-08 18:55:44,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:44,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:44,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:44,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 087e27baac917de6bad03c6444a74b6d, NAME => 'testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 18:55:44,382 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=924cd1dc6b0366a92e4bc7c5a4762a13, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:44,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:44,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:44,383 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301744382"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301744382"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301744382"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301744382"}]},"ts":"1678301744382"} 2023-03-08 18:55:44,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:44,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:44,385 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-03-08 18:55:44,385 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure 924cd1dc6b0366a92e4bc7c5a4762a13, server=jenkins-hbase6.apache.org,46397,1678301621627 in 315 msec 2023-03-08 18:55:44,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=924cd1dc6b0366a92e4bc7c5a4762a13, ASSIGN in 473 msec 2023-03-08 18:55:44,386 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:44,387 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1 2023-03-08 18:55:44,387 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1 2023-03-08 18:55:44,387 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 087e27baac917de6bad03c6444a74b6d columnFamilyName family1 2023-03-08 18:55:44,394 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-1bf5917d1e08401eb5729ee7799742d6 2023-03-08 18:55:44,398 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-5d0f81d8cf7f4498add583a785c93576 2023-03-08 18:55:44,404 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-8e47fa9593bd4665acef00d8eb428147 2023-03-08 18:55:44,409 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-fe3a899f5d434fb7b3863d0a7fe8cfb8 2023-03-08 18:55:44,409 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(310): Store=087e27baac917de6bad03c6444a74b6d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:44,410 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:44,411 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2 2023-03-08 18:55:44,411 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2 2023-03-08 18:55:44,411 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 087e27baac917de6bad03c6444a74b6d columnFamilyName family2 2023-03-08 18:55:44,421 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-3f86ffff34c0468fb5aa1b8d80d7fc87 2023-03-08 18:55:44,427 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-9fd316deb3434201a8533d6648b1eed9 2023-03-08 18:55:44,432 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-a154990d75dd4a3681e935917dce1002 2023-03-08 18:55:44,438 DEBUG [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(539): loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-d39edff1a8db45eb9fb524f0d121653d 2023-03-08 18:55:44,438 INFO [StoreOpener-087e27baac917de6bad03c6444a74b6d-1] regionserver.HStore(310): Store=087e27baac917de6bad03c6444a74b6d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:44,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:44,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:44,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:44,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:55:44,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 087e27baac917de6bad03c6444a74b6d; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69147769, jitterRate=0.030382052063941956}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:44,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:44,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d., pid=101, masterSystemTime=1678301744219 2023-03-08 18:55:44,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-08 18:55:44,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 18:55:44,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:44,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:44,445 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=087e27baac917de6bad03c6444a74b6d, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:44,445 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678301744445"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301744445"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301744445"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301744445"}]},"ts":"1678301744445"} 2023-03-08 18:55:44,448 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-03-08 18:55:44,448 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 087e27baac917de6bad03c6444a74b6d, server=jenkins-hbase6.apache.org,46397,1678301621627 in 379 msec 2023-03-08 18:55:44,449 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-03-08 18:55:44,449 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=087e27baac917de6bad03c6444a74b6d, ASSIGN in 536 msec 2023-03-08 18:55:44,450 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301744449"}]},"ts":"1678301744449"} 2023-03-08 18:55:44,450 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-08 18:55:44,452 INFO [PEWorker-2] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-08 18:55:44,453 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 550 msec 2023-03-08 18:55:44,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 18:55:44,512 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-08 18:55:44,525 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:44,532 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:44,540 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:44,541 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:44,547 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 18:55:44,560 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-08 18:55:44,560 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:44,560 DEBUG [Listener at localhost/36281] zookeeper.ReadOnlyZKClient(139): Connect 0x0c8ddbb5 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:55:44,565 DEBUG [Listener at localhost/36281] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32f4cb3b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:55:44,565 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:44,565 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:44,565 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:45,039 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308699e087f78ad4e35942da8694e40748d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308699e087f78ad4e35942da8694e40748d 2023-03-08 18:55:45,040 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:45,052 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:55:45,056 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:49732, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:55:45,062 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:55:45,065 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:50952, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:55:45,075 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/8cbc43de08084c7a8e39aab3fb95cf64 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-08 18:55:45,076 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/8cbc43de08084c7a8e39aab3fb95cf64}] 2023-03-08 18:55:45,082 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/8cbc43de08084c7a8e39aab3fb95cf64 for inclusion in 087e27baac917de6bad03c6444a74b6d/family1 2023-03-08 18:55:45,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-08 18:55:45,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 18:55:45,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:45,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/8cbc43de08084c7a8e39aab3fb95cf64 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__q4m49f8ds4kq8phgthuknqlb5d581n8r72o185kk134mf07fu9kn0ui4f0u9fnh0/family1/8cbc43de08084c7a8e39aab3fb95cf64 2023-03-08 18:55:45,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__q4m49f8ds4kq8phgthuknqlb5d581n8r72o185kk134mf07fu9kn0ui4f0u9fnh0/family1/8cbc43de08084c7a8e39aab3fb95cf64 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/f46e939f4e894b5c81994a6a35cebead_SeqId_34_ 2023-03-08 18:55:45,088 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__q4m49f8ds4kq8phgthuknqlb5d581n8r72o185kk134mf07fu9kn0ui4f0u9fnh0/family1/8cbc43de08084c7a8e39aab3fb95cf64 into 087e27baac917de6bad03c6444a74b6d/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/f46e939f4e894b5c81994a6a35cebead_SeqId_34_ - updating store file list. 2023-03-08 18:55:45,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/f46e939f4e894b5c81994a6a35cebead_SeqId_34_ into 087e27baac917de6bad03c6444a74b6d/family1 2023-03-08 18:55:45,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__q4m49f8ds4kq8phgthuknqlb5d581n8r72o185kk134mf07fu9kn0ui4f0u9fnh0/family1/8cbc43de08084c7a8e39aab3fb95cf64 into 087e27baac917de6bad03c6444a74b6d/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/f46e939f4e894b5c81994a6a35cebead_SeqId_34_) 2023-03-08 18:55:45,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__q4m49f8ds4kq8phgthuknqlb5d581n8r72o185kk134mf07fu9kn0ui4f0u9fnh0/family1/8cbc43de08084c7a8e39aab3fb95cf64 2023-03-08 18:55:45,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:45,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-08 18:55:45,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(1912): 087e27baac917de6bad03c6444a74b6d/family1 is initiating minor compaction (all files) 2023-03-08 18:55:45,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6ddc0886; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-08 18:55:45,095 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:45,096 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:45,097 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303087f22d3d955f34ee1bb6c816ddd2b0323 2023-03-08 18:55:45,098 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308bd38f4ddb17e4ba0ab4caf2e020d94db 2023-03-08 18:55:45,099 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308e031c0ffb72d4592b6b7d045ca4da1f1 2023-03-08 18:55:45,100 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308eb2caa19062e42bc9b4c840730ae41b1 2023-03-08 18:55:45,100 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 18:55:45,100 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 18:55:45,100 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:45,440 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308c9f5231987914439922504664e7bc986 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c9f5231987914439922504664e7bc986 2023-03-08 18:55:45,441 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308f66a6e7f9aac4dd1b2eba59fa3444491 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308f66a6e7f9aac4dd1b2eba59fa3444491 2023-03-08 18:55:45,441 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:45,442 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:45,483 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/0e7e2dd4295943729f125107d23b1ad8 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-08 18:55:45,483 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/2e6bd129764048e1ad1c8b5c2a362039 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-08 18:55:45,485 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=32 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/2e6bd129764048e1ad1c8b5c2a362039}] 2023-03-08 18:55:45,485 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/0e7e2dd4295943729f125107d23b1ad8}] 2023-03-08 18:55:45,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/2e6bd129764048e1ad1c8b5c2a362039 for inclusion in 70855e40ae020380397c6a9b2ff4c892/family1 2023-03-08 18:55:45,488 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/0e7e2dd4295943729f125107d23b1ad8 for inclusion in 924cd1dc6b0366a92e4bc7c5a4762a13/family1 2023-03-08 18:55:45,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-08 18:55:45,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 18:55:45,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-08 18:55:45,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:45,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 18:55:45,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:45,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/2e6bd129764048e1ad1c8b5c2a362039 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__tlul09faa1t1176s07ocfrm73gmtknvca95gn1mpdl0vua6573rr9nphtdltavpe/family1/2e6bd129764048e1ad1c8b5c2a362039 2023-03-08 18:55:45,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/0e7e2dd4295943729f125107d23b1ad8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__jmhbs90k4d0g8coe9ghdjhooqdp7cgq4gfcm06hbfpi7vupoqi9v5uitvf7ic1ck/family1/0e7e2dd4295943729f125107d23b1ad8 2023-03-08 18:55:45,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__tlul09faa1t1176s07ocfrm73gmtknvca95gn1mpdl0vua6573rr9nphtdltavpe/family1/2e6bd129764048e1ad1c8b5c2a362039 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/909c70c0d0c44968ad63e7f2cf872d55_SeqId_34_ 2023-03-08 18:55:45,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__jmhbs90k4d0g8coe9ghdjhooqdp7cgq4gfcm06hbfpi7vupoqi9v5uitvf7ic1ck/family1/0e7e2dd4295943729f125107d23b1ad8 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_ 2023-03-08 18:55:45,495 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__jmhbs90k4d0g8coe9ghdjhooqdp7cgq4gfcm06hbfpi7vupoqi9v5uitvf7ic1ck/family1/0e7e2dd4295943729f125107d23b1ad8 into 924cd1dc6b0366a92e4bc7c5a4762a13/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_ - updating store file list. 2023-03-08 18:55:45,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__tlul09faa1t1176s07ocfrm73gmtknvca95gn1mpdl0vua6573rr9nphtdltavpe/family1/2e6bd129764048e1ad1c8b5c2a362039 into 70855e40ae020380397c6a9b2ff4c892/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/909c70c0d0c44968ad63e7f2cf872d55_SeqId_34_ - updating store file list. 2023-03-08 18:55:45,499 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_ into 924cd1dc6b0366a92e4bc7c5a4762a13/family1 2023-03-08 18:55:45,499 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__jmhbs90k4d0g8coe9ghdjhooqdp7cgq4gfcm06hbfpi7vupoqi9v5uitvf7ic1ck/family1/0e7e2dd4295943729f125107d23b1ad8 into 924cd1dc6b0366a92e4bc7c5a4762a13/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_) 2023-03-08 18:55:45,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/909c70c0d0c44968ad63e7f2cf872d55_SeqId_34_ into 70855e40ae020380397c6a9b2ff4c892/family1 2023-03-08 18:55:45,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__tlul09faa1t1176s07ocfrm73gmtknvca95gn1mpdl0vua6573rr9nphtdltavpe/family1/2e6bd129764048e1ad1c8b5c2a362039 into 70855e40ae020380397c6a9b2ff4c892/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/909c70c0d0c44968ad63e7f2cf872d55_SeqId_34_) 2023-03-08 18:55:45,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__jmhbs90k4d0g8coe9ghdjhooqdp7cgq4gfcm06hbfpi7vupoqi9v5uitvf7ic1ck/family1/0e7e2dd4295943729f125107d23b1ad8 2023-03-08 18:55:45,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testCompactionWithHFileLink__tlul09faa1t1176s07ocfrm73gmtknvca95gn1mpdl0vua6573rr9nphtdltavpe/family1/2e6bd129764048e1ad1c8b5c2a362039 2023-03-08 18:55:45,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:45,502 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:45,502 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:45,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-08 18:55:45,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(1912): 924cd1dc6b0366a92e4bc7c5a4762a13/family1 is initiating minor compaction (all files) 2023-03-08 18:55:45,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@e6b87e9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-03-08 18:55:45,504 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303084375b7c440154a26a7dc9e282a5e18b9 2023-03-08 18:55:45,505 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308713c7555f98f4fa0ba385fa71c691361 2023-03-08 18:55:45,505 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:45,506 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:45,506 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308727ab334edb849828438b3186c104c18 2023-03-08 18:55:45,507 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308e3e4d6fb2fb540bc957fd38589dd1e8f 2023-03-08 18:55:45,508 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 18:55:45,508 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303080938889a758f4e61bf4ad48979d0475b 2023-03-08 18:55:45,508 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 18:55:45,508 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:45,509 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308be157a9fc9f34b76abcdcd08589e9612 2023-03-08 18:55:45,509 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308bf706d2daa604ae4886e5d58be05e7ae 2023-03-08 18:55:45,510 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308d911cdc9c57443b69a0787987d067992 2023-03-08 18:55:45,510 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 18:55:45,510 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 18:55:45,511 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 18:55:45,511 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 18:55:45,513 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:45,515 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:45,518 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:45,519 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:45,524 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:45,524 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:45,543 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=285 (was 255) - Thread LEAK? -, OpenFileDescriptor=765 (was 648) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=139 (was 150), ProcessCount=170 (was 172), AvailableMemoryMB=6901 (was 7037) 2023-03-08 18:55:45,558 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=285, OpenFileDescriptor=765, MaxFileDescriptor=60000, SystemLoadAverage=139, ProcessCount=170, AvailableMemoryMB=6900 2023-03-08 18:55:45,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$15(3014): Client=jenkins//172.31.2.20 creating {NAME => 'ns'} 2023-03-08 18:55:45,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-08 18:55:45,567 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family1/f9dd83c62b4a40f2bf7a1b1882d611de as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/f9dd83c62b4a40f2bf7a1b1882d611de 2023-03-08 18:55:45,567 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/.tmp/family2/e1e26385b4b546e3b6a85fa211af4514 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/e1e26385b4b546e3b6a85fa211af4514 2023-03-08 18:55:45,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-08 18:55:45,572 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 18:55:45,572 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 18:55:45,572 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 18:55:45,573 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 70855e40ae020380397c6a9b2ff4c892/family2 of 70855e40ae020380397c6a9b2ff4c892 into e1e26385b4b546e3b6a85fa211af4514(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:45,573 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 70855e40ae020380397c6a9b2ff4c892/family1 of 70855e40ae020380397c6a9b2ff4c892 into f9dd83c62b4a40f2bf7a1b1882d611de(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:45,573 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:45,573 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:55:45,573 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892., storeName=70855e40ae020380397c6a9b2ff4c892/family2, priority=12, startTime=1678301744279; duration=1sec 2023-03-08 18:55:45,573 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892., storeName=70855e40ae020380397c6a9b2ff4c892/family1, priority=12, startTime=1678301744279; duration=1sec 2023-03-08 18:55:45,573 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-03-08 18:55:45,574 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-08 18:55:45,574 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 087e27baac917de6bad03c6444a74b6d/family1 in testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:45,574 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 924cd1dc6b0366a92e4bc7c5a4762a13/family1 in testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:45,574 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-8e47fa9593bd4665acef00d8eb428147, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-fe3a899f5d434fb7b3863d0a7fe8cfb8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-5d0f81d8cf7f4498add583a785c93576, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-1bf5917d1e08401eb5729ee7799742d6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/f46e939f4e894b5c81994a6a35cebead_SeqId_34_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp, totalSize=28.9 K 2023-03-08 18:55:45,574 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-a7594ec12b084addbcc1369ec9928778, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-00a3dc9d388b42da8cd5f72975734740, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-49bf9da251214ae7b2178764443f2a22, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-b83ef318305a4c50852898001a0735b8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp, totalSize=28.9 K 2023-03-08 18:55:45,575 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 10 msec 2023-03-08 18:55:45,575 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-8e47fa9593bd4665acef00d8eb428147, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719978 2023-03-08 18:55:45,575 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-a7594ec12b084addbcc1369ec9928778, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719975 2023-03-08 18:55:45,575 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-fe3a899f5d434fb7b3863d0a7fe8cfb8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721767 2023-03-08 18:55:45,575 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-00a3dc9d388b42da8cd5f72975734740, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721767 2023-03-08 18:55:45,576 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-5d0f81d8cf7f4498add583a785c93576, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723168 2023-03-08 18:55:45,576 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-49bf9da251214ae7b2178764443f2a22, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723168 2023-03-08 18:55:45,576 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-1bf5917d1e08401eb5729ee7799742d6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724959 2023-03-08 18:55:45,576 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-b83ef318305a4c50852898001a0735b8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724959 2023-03-08 18:55:45,577 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting f46e939f4e894b5c81994a6a35cebead_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678301719978 2023-03-08 18:55:45,577 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678301719975 2023-03-08 18:55:45,588 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-1f9fdb228c8647fb87719f5cd2ad1bc0, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-c8c661ff321b4588b2eb3fea2a3e6a20, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-bd3164ac449643289a677b6bf46c83f9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-11fe3fe861b94ba28c3c71d689725dd4] to archive 2023-03-08 18:55:45,589 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:45,590 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] throttle.PressureAwareThroughputController(145): 087e27baac917de6bad03c6444a74b6d#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:45,591 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-1f9fdb228c8647fb87719f5cd2ad1bc0 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-1f9fdb228c8647fb87719f5cd2ad1bc0 2023-03-08 18:55:45,591 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 924cd1dc6b0366a92e4bc7c5a4762a13#family1#compaction#117 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:45,592 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-c8c661ff321b4588b2eb3fea2a3e6a20 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-c8c661ff321b4588b2eb3fea2a3e6a20 2023-03-08 18:55:45,593 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-bd3164ac449643289a677b6bf46c83f9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-bd3164ac449643289a677b6bf46c83f9 2023-03-08 18:55:45,594 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-11fe3fe861b94ba28c3c71d689725dd4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family1/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-11fe3fe861b94ba28c3c71d689725dd4 2023-03-08 18:55:45,596 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-8cf2aef7d2a74556b2cb4eb00f828dba, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-2ea9681be9bb4ce8b6811c960e6d58c7, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-4837d54c0e78408ab1fa203e87fa07e9, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-6caa146fb006423983892ce72dc16ee3] to archive 2023-03-08 18:55:45,596 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:45,597 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-8cf2aef7d2a74556b2cb4eb00f828dba to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-8cf2aef7d2a74556b2cb4eb00f828dba 2023-03-08 18:55:45,599 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-2ea9681be9bb4ce8b6811c960e6d58c7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-2ea9681be9bb4ce8b6811c960e6d58c7 2023-03-08 18:55:45,600 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-4837d54c0e78408ab1fa203e87fa07e9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-4837d54c0e78408ab1fa203e87fa07e9 2023-03-08 18:55:45,601 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-6caa146fb006423983892ce72dc16ee3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/family2/testCompactionWithHFileLink=70855e40ae020380397c6a9b2ff4c892-6caa146fb006423983892ce72dc16ee3 2023-03-08 18:55:45,605 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family1/621ca17bb1734d12af6817b8c8bbf2db as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/621ca17bb1734d12af6817b8c8bbf2db 2023-03-08 18:55:45,608 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 924cd1dc6b0366a92e4bc7c5a4762a13/family1 of 924cd1dc6b0366a92e4bc7c5a4762a13 into 621ca17bb1734d12af6817b8c8bbf2db(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:55:45,608 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:45,608 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678301745503; duration=0sec 2023-03-08 18:55:45,608 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 18:55:45,608 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 18:55:45,610 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 18:55:45,610 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1912): 924cd1dc6b0366a92e4bc7c5a4762a13/family2 is initiating minor compaction (all files) 2023-03-08 18:55:45,610 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 924cd1dc6b0366a92e4bc7c5a4762a13/family2 in testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:55:45,610 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-cdcbd35b856741ea902048713afcc63a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-9cc8be2569144b7f8f627c4dfd4587dc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-451c93d932004dc6a9e72401f6bc0ae6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-bb018b416cee4b28b129da16edfbb5d3] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp, totalSize=20.4 K 2023-03-08 18:55:45,611 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-cdcbd35b856741ea902048713afcc63a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719975 2023-03-08 18:55:45,611 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-9cc8be2569144b7f8f627c4dfd4587dc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721767 2023-03-08 18:55:45,611 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-451c93d932004dc6a9e72401f6bc0ae6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723168 2023-03-08 18:55:45,612 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-bb018b416cee4b28b129da16edfbb5d3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724959 2023-03-08 18:55:45,621 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): 924cd1dc6b0366a92e4bc7c5a4762a13#family2#compaction#118 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:45,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-08 18:55:45,673 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 18:55:45,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-08 18:55:45,675 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:55:45,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-08 18:55:45,676 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:55:45,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 18:55:45,678 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:45,678 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:45,678 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:45,678 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e empty. 2023-03-08 18:55:45,678 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b empty. 2023-03-08 18:55:45,678 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1 empty. 2023-03-08 18:55:45,678 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:45,678 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:45,679 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:45,679 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-08 18:55:45,687 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-a7594ec12b084addbcc1369ec9928778, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-00a3dc9d388b42da8cd5f72975734740, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-49bf9da251214ae7b2178764443f2a22, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-b83ef318305a4c50852898001a0735b8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_] to archive 2023-03-08 18:55:45,688 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:45,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-a7594ec12b084addbcc1369ec9928778 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-a7594ec12b084addbcc1369ec9928778 2023-03-08 18:55:45,690 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-00a3dc9d388b42da8cd5f72975734740 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-00a3dc9d388b42da8cd5f72975734740 2023-03-08 18:55:45,691 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-49bf9da251214ae7b2178764443f2a22 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-49bf9da251214ae7b2178764443f2a22 2023-03-08 18:55:45,693 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-b83ef318305a4c50852898001a0735b8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-b83ef318305a4c50852898001a0735b8 2023-03-08 18:55:45,694 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family1/f2b95e3e47bd48ae8147986dc9accd74_SeqId_34_ 2023-03-08 18:55:45,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 18:55:45,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 18:55:46,086 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-08 18:55:46,087 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 8eb59701ce5a41a0784e567680dbb06b, NAME => 'ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:55:46,087 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => b4625189889702b73dd8f5a775ba90f1, NAME => 'ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:55:46,087 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => d2ad469b39f67aef7d6b850cf65d8a4e, NAME => 'ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:55:46,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 18:55:46,492 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:46,493 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 8eb59701ce5a41a0784e567680dbb06b, disabling compactions & flushes 2023-03-08 18:55:46,493 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:46,493 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:46,493 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. after waiting 0 ms 2023-03-08 18:55:46,493 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:46,493 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:46,493 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:55:46,496 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:46,496 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:46,496 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing b4625189889702b73dd8f5a775ba90f1, disabling compactions & flushes 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing d2ad469b39f67aef7d6b850cf65d8a4e, disabling compactions & flushes 2023-03-08 18:55:46,497 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:46,497 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. after waiting 0 ms 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. after waiting 0 ms 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:46,497 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:46,497 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:55:46,497 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:46,499 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:55:46,500 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678301746500"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301746500"}]},"ts":"1678301746500"} 2023-03-08 18:55:46,500 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678301746500"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301746500"}]},"ts":"1678301746500"} 2023-03-08 18:55:46,500 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678301746500"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301746500"}]},"ts":"1678301746500"} 2023-03-08 18:55:46,501 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:55:46,501 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 18:55:46,502 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:55:46,502 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301746502"}]},"ts":"1678301746502"} 2023-03-08 18:55:46,503 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-08 18:55:46,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d2ad469b39f67aef7d6b850cf65d8a4e, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b4625189889702b73dd8f5a775ba90f1, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8eb59701ce5a41a0784e567680dbb06b, ASSIGN}] 2023-03-08 18:55:46,509 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d2ad469b39f67aef7d6b850cf65d8a4e, ASSIGN 2023-03-08 18:55:46,509 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b4625189889702b73dd8f5a775ba90f1, ASSIGN 2023-03-08 18:55:46,509 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8eb59701ce5a41a0784e567680dbb06b, ASSIGN 2023-03-08 18:55:46,509 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d2ad469b39f67aef7d6b850cf65d8a4e, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:55:46,509 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b4625189889702b73dd8f5a775ba90f1, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:55:46,510 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8eb59701ce5a41a0784e567680dbb06b, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:55:46,662 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=b4625189889702b73dd8f5a775ba90f1, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:46,662 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=8eb59701ce5a41a0784e567680dbb06b, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:46,662 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d2ad469b39f67aef7d6b850cf65d8a4e, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:46,662 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678301746661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301746661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301746661"}]},"ts":"1678301746661"} 2023-03-08 18:55:46,662 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678301746661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301746661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301746661"}]},"ts":"1678301746661"} 2023-03-08 18:55:46,662 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678301746661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301746661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301746661"}]},"ts":"1678301746661"} 2023-03-08 18:55:46,663 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure b4625189889702b73dd8f5a775ba90f1, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:46,664 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=108, state=RUNNABLE; OpenRegionProcedure 8eb59701ce5a41a0784e567680dbb06b, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:46,664 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure d2ad469b39f67aef7d6b850cf65d8a4e, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:55:46,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 18:55:46,807 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family1/793207ae2a0441378c0ac9ee58417cae as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/793207ae2a0441378c0ac9ee58417cae 2023-03-08 18:55:46,810 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 087e27baac917de6bad03c6444a74b6d/family1 of 087e27baac917de6bad03c6444a74b6d into 793207ae2a0441378c0ac9ee58417cae(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:46,811 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:46,811 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678301745093; duration=1sec 2023-03-08 18:55:46,811 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-08 18:55:46,811 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 18:55:46,813 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 18:55:46,813 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HStore(1912): 087e27baac917de6bad03c6444a74b6d/family2 is initiating minor compaction (all files) 2023-03-08 18:55:46,813 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 087e27baac917de6bad03c6444a74b6d/family2 in testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:55:46,813 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-3f86ffff34c0468fb5aa1b8d80d7fc87, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-a154990d75dd4a3681e935917dce1002, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-d39edff1a8db45eb9fb524f0d121653d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-9fd316deb3434201a8533d6648b1eed9] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp, totalSize=20.4 K 2023-03-08 18:55:46,813 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-3f86ffff34c0468fb5aa1b8d80d7fc87, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301719978 2023-03-08 18:55:46,814 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-a154990d75dd4a3681e935917dce1002, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301721767 2023-03-08 18:55:46,814 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-d39edff1a8db45eb9fb524f0d121653d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301723168 2023-03-08 18:55:46,815 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-9fd316deb3434201a8533d6648b1eed9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301724959 2023-03-08 18:55:46,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:46,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d2ad469b39f67aef7d6b850cf65d8a4e, NAME => 'ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 18:55:46,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:46,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:46,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:46,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:46,820 INFO [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:46,821 DEBUG [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1 2023-03-08 18:55:46,821 DEBUG [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1 2023-03-08 18:55:46,821 INFO [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2ad469b39f67aef7d6b850cf65d8a4e columnFamilyName family1 2023-03-08 18:55:46,823 INFO [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] regionserver.HStore(310): Store=d2ad469b39f67aef7d6b850cf65d8a4e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:46,823 INFO [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:46,824 DEBUG [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2 2023-03-08 18:55:46,824 DEBUG [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2 2023-03-08 18:55:46,824 INFO [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2ad469b39f67aef7d6b850cf65d8a4e columnFamilyName family2 2023-03-08 18:55:46,826 INFO [StoreOpener-d2ad469b39f67aef7d6b850cf65d8a4e-1] regionserver.HStore(310): Store=d2ad469b39f67aef7d6b850cf65d8a4e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:46,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:46,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:46,828 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] throttle.PressureAwareThroughputController(145): 087e27baac917de6bad03c6444a74b6d#family2#compaction#119 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:46,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:46,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:55:46,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:55:46,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened d2ad469b39f67aef7d6b850cf65d8a4e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61104173, jitterRate=-0.08947686851024628}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:46,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:46,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e., pid=111, masterSystemTime=1678301746815 2023-03-08 18:55:46,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:46,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:46,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:46,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8eb59701ce5a41a0784e567680dbb06b, NAME => 'ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 18:55:46,837 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d2ad469b39f67aef7d6b850cf65d8a4e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:46,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:46,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:46,837 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678301746837"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301746837"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301746837"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301746837"}]},"ts":"1678301746837"} 2023-03-08 18:55:46,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:46,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:46,837 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/.tmp/family2/de528b09c1174c67ac8995f2fa81b453 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/de528b09c1174c67ac8995f2fa81b453 2023-03-08 18:55:46,838 INFO [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:46,839 DEBUG [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1 2023-03-08 18:55:46,839 DEBUG [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1 2023-03-08 18:55:46,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=106 2023-03-08 18:55:46,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure d2ad469b39f67aef7d6b850cf65d8a4e, server=jenkins-hbase6.apache.org,46397,1678301621627 in 174 msec 2023-03-08 18:55:46,840 INFO [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8eb59701ce5a41a0784e567680dbb06b columnFamilyName family1 2023-03-08 18:55:46,840 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d2ad469b39f67aef7d6b850cf65d8a4e, ASSIGN in 333 msec 2023-03-08 18:55:46,841 INFO [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] regionserver.HStore(310): Store=8eb59701ce5a41a0784e567680dbb06b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:46,841 INFO [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:46,842 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 924cd1dc6b0366a92e4bc7c5a4762a13/family2 of 924cd1dc6b0366a92e4bc7c5a4762a13 into de528b09c1174c67ac8995f2fa81b453(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:46,842 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:55:46,842 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13., storeName=924cd1dc6b0366a92e4bc7c5a4762a13/family2, priority=12, startTime=1678301744381; duration=1sec 2023-03-08 18:55:46,842 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-08 18:55:46,842 DEBUG [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2 2023-03-08 18:55:46,842 DEBUG [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2 2023-03-08 18:55:46,842 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-08 18:55:46,842 INFO [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8eb59701ce5a41a0784e567680dbb06b columnFamilyName family2 2023-03-08 18:55:46,842 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 18:55:46,842 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 18:55:46,842 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. because compaction request was cancelled 2023-03-08 18:55:46,842 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-08 18:55:46,843 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 18:55:46,843 INFO [StoreOpener-8eb59701ce5a41a0784e567680dbb06b-1] regionserver.HStore(310): Store=8eb59701ce5a41a0784e567680dbb06b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:46,843 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 18:55:46,843 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. because compaction request was cancelled 2023-03-08 18:55:46,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:46,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:46,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:46,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:55:46,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:55:46,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 8eb59701ce5a41a0784e567680dbb06b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64500835, jitterRate=-0.03886266052722931}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:46,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:55:46,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b., pid=110, masterSystemTime=1678301746815 2023-03-08 18:55:46,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:46,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:46,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:46,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b4625189889702b73dd8f5a775ba90f1, NAME => 'ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 18:55:46,849 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=8eb59701ce5a41a0784e567680dbb06b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:46,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:46,850 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678301746849"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301746849"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301746849"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301746849"}]},"ts":"1678301746849"} 2023-03-08 18:55:46,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:55:46,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:46,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:46,851 INFO [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:46,852 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=108 2023-03-08 18:55:46,852 DEBUG [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1 2023-03-08 18:55:46,852 DEBUG [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1 2023-03-08 18:55:46,852 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=108, state=SUCCESS; OpenRegionProcedure 8eb59701ce5a41a0784e567680dbb06b, server=jenkins-hbase6.apache.org,46397,1678301621627 in 187 msec 2023-03-08 18:55:46,852 INFO [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b4625189889702b73dd8f5a775ba90f1 columnFamilyName family1 2023-03-08 18:55:46,853 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8eb59701ce5a41a0784e567680dbb06b, ASSIGN in 345 msec 2023-03-08 18:55:46,853 INFO [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] regionserver.HStore(310): Store=b4625189889702b73dd8f5a775ba90f1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:46,853 INFO [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:46,854 DEBUG [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2 2023-03-08 18:55:46,854 DEBUG [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2 2023-03-08 18:55:46,854 INFO [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b4625189889702b73dd8f5a775ba90f1 columnFamilyName family2 2023-03-08 18:55:46,855 INFO [StoreOpener-b4625189889702b73dd8f5a775ba90f1-1] regionserver.HStore(310): Store=b4625189889702b73dd8f5a775ba90f1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:55:46,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:46,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:46,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:55:46,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:55:46,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:55:46,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened b4625189889702b73dd8f5a775ba90f1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74427864, jitterRate=0.109061598777771}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:55:46,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:55:46,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1., pid=109, masterSystemTime=1678301746815 2023-03-08 18:55:46,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:46,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:46,861 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=b4625189889702b73dd8f5a775ba90f1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:46,861 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678301746860"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301746860"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301746860"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301746860"}]},"ts":"1678301746860"} 2023-03-08 18:55:46,863 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=107 2023-03-08 18:55:46,863 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure b4625189889702b73dd8f5a775ba90f1, server=jenkins-hbase6.apache.org,46397,1678301621627 in 199 msec 2023-03-08 18:55:46,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-03-08 18:55:46,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b4625189889702b73dd8f5a775ba90f1, ASSIGN in 356 msec 2023-03-08 18:55:46,864 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:55:46,864 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301746864"}]},"ts":"1678301746864"} 2023-03-08 18:55:46,865 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-08 18:55:46,867 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:55:46,868 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1940 sec 2023-03-08 18:55:46,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-cdcbd35b856741ea902048713afcc63a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-9cc8be2569144b7f8f627c4dfd4587dc, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-451c93d932004dc6a9e72401f6bc0ae6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-bb018b416cee4b28b129da16edfbb5d3] to archive 2023-03-08 18:55:46,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:46,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-cdcbd35b856741ea902048713afcc63a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-cdcbd35b856741ea902048713afcc63a 2023-03-08 18:55:46,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-9cc8be2569144b7f8f627c4dfd4587dc to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-9cc8be2569144b7f8f627c4dfd4587dc 2023-03-08 18:55:46,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-451c93d932004dc6a9e72401f6bc0ae6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-451c93d932004dc6a9e72401f6bc0ae6 2023-03-08 18:55:46,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-bb018b416cee4b28b129da16edfbb5d3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/family2/testCompactionWithHFileLink=924cd1dc6b0366a92e4bc7c5a4762a13-bb018b416cee4b28b129da16edfbb5d3 2023-03-08 18:55:46,887 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-8e47fa9593bd4665acef00d8eb428147, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-fe3a899f5d434fb7b3863d0a7fe8cfb8, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-5d0f81d8cf7f4498add583a785c93576, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-1bf5917d1e08401eb5729ee7799742d6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/f46e939f4e894b5c81994a6a35cebead_SeqId_34_] to archive 2023-03-08 18:55:46,887 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:46,888 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-8e47fa9593bd4665acef00d8eb428147 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-8e47fa9593bd4665acef00d8eb428147 2023-03-08 18:55:46,889 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-fe3a899f5d434fb7b3863d0a7fe8cfb8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-fe3a899f5d434fb7b3863d0a7fe8cfb8 2023-03-08 18:55:46,890 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-5d0f81d8cf7f4498add583a785c93576 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-5d0f81d8cf7f4498add583a785c93576 2023-03-08 18:55:46,891 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-1bf5917d1e08401eb5729ee7799742d6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-1bf5917d1e08401eb5729ee7799742d6 2023-03-08 18:55:46,892 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/f46e939f4e894b5c81994a6a35cebead_SeqId_34_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family1/f46e939f4e894b5c81994a6a35cebead_SeqId_34_ 2023-03-08 18:55:47,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 18:55:47,781 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-08 18:55:47,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:47,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:47,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:47,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:47,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 18:55:47,792 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 18:55:47,792 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:47,792 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 18:55:47,792 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 18:55:47,792 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:47,792 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:47,793 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:47,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:47,793 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:47,794 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:47,794 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:47,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:47,795 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 18:55:47,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 18:55:47,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 18:55:47,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:47,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 18:55:47,795 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:47,795 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. started... 2023-03-08 18:55:47,797 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2ad469b39f67aef7d6b850cf65d8a4e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:55:47,797 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:47,797 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:47,797 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:47,797 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:47,797 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. started... 2023-03-08 18:55:47,797 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. started... 2023-03-08 18:55:47,797 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8eb59701ce5a41a0784e567680dbb06b 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:55:47,797 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b4625189889702b73dd8f5a775ba90f1 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:55:48,048 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/.tmp/family2/18657c2b25924a1a88c6c3420563a63f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/18657c2b25924a1a88c6c3420563a63f 2023-03-08 18:55:48,052 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 087e27baac917de6bad03c6444a74b6d/family2 of 087e27baac917de6bad03c6444a74b6d into 18657c2b25924a1a88c6c3420563a63f(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:48,052 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:55:48,052 INFO [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d., storeName=087e27baac917de6bad03c6444a74b6d/family2, priority=12, startTime=1678301744443; duration=1sec 2023-03-08 18:55:48,052 DEBUG [RS:0;jenkins-hbase6:46397-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:55:48,087 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-3f86ffff34c0468fb5aa1b8d80d7fc87, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-a154990d75dd4a3681e935917dce1002, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-d39edff1a8db45eb9fb524f0d121653d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-9fd316deb3434201a8533d6648b1eed9] to archive 2023-03-08 18:55:48,088 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:48,089 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-3f86ffff34c0468fb5aa1b8d80d7fc87 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-3f86ffff34c0468fb5aa1b8d80d7fc87 2023-03-08 18:55:48,090 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-a154990d75dd4a3681e935917dce1002 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-a154990d75dd4a3681e935917dce1002 2023-03-08 18:55:48,091 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-d39edff1a8db45eb9fb524f0d121653d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-d39edff1a8db45eb9fb524f0d121653d 2023-03-08 18:55:48,092 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-9fd316deb3434201a8533d6648b1eed9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/family2/testCompactionWithHFileLink=087e27baac917de6bad03c6444a74b6d-9fd316deb3434201a8533d6648b1eed9 2023-03-08 18:55:48,212 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308d221c789f1bf41feba96deaa9606f6f2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308d221c789f1bf41feba96deaa9606f6f2 2023-03-08 18:55:48,212 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308ca4e82da78564c4c9b881c08bcecdd79 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ca4e82da78564c4c9b881c08bcecdd79 2023-03-08 18:55:48,213 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303084d0870d0a6c64d388c89afb4aceda2c2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303084d0870d0a6c64d388c89afb4aceda2c2 2023-03-08 18:55:48,218 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/7f8309375eb14472adf0d6ee1c31d6d4 2023-03-08 18:55:48,218 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/6aeec20bf44542a782ed2d04b28ea7a1 2023-03-08 18:55:48,218 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/f83428eea1624a94b0ed90565aafc027 2023-03-08 18:55:48,648 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030883e53d02d76f49cd8e7c8ae0db3a8f38 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030883e53d02d76f49cd8e7c8ae0db3a8f38 2023-03-08 18:55:48,648 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303087e1f9b01a91647418d64218765765226 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303087e1f9b01a91647418d64218765765226 2023-03-08 18:55:48,648 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308927648425b084d75b93565356b8f57ad to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308927648425b084d75b93565356b8f57ad 2023-03-08 18:55:49,054 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/ea67bef51fc14a3d9278d9ca60ddfbff 2023-03-08 18:55:49,055 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/6835c1db1860462dae97ca0d022af34b 2023-03-08 18:55:49,055 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/29edb21579114693b285fe61de1810aa 2023-03-08 18:55:49,059 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/7f8309375eb14472adf0d6ee1c31d6d4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7f8309375eb14472adf0d6ee1c31d6d4 2023-03-08 18:55:49,059 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/6aeec20bf44542a782ed2d04b28ea7a1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/6aeec20bf44542a782ed2d04b28ea7a1 2023-03-08 18:55:49,059 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/f83428eea1624a94b0ed90565aafc027 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f83428eea1624a94b0ed90565aafc027 2023-03-08 18:55:49,063 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f83428eea1624a94b0ed90565aafc027, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 18:55:49,063 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7f8309375eb14472adf0d6ee1c31d6d4, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 18:55:49,064 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/6aeec20bf44542a782ed2d04b28ea7a1, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 18:55:49,064 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/6835c1db1860462dae97ca0d022af34b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/6835c1db1860462dae97ca0d022af34b 2023-03-08 18:55:49,064 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/ea67bef51fc14a3d9278d9ca60ddfbff as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/ea67bef51fc14a3d9278d9ca60ddfbff 2023-03-08 18:55:49,065 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/29edb21579114693b285fe61de1810aa as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/29edb21579114693b285fe61de1810aa 2023-03-08 18:55:49,067 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/6835c1db1860462dae97ca0d022af34b, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:55:49,067 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/ea67bef51fc14a3d9278d9ca60ddfbff, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:55:49,068 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 8eb59701ce5a41a0784e567680dbb06b in 1271ms, sequenceid=5, compaction requested=false 2023-03-08 18:55:49,068 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/29edb21579114693b285fe61de1810aa, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:55:49,068 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b4625189889702b73dd8f5a775ba90f1 in 1271ms, sequenceid=5, compaction requested=false 2023-03-08 18:55:49,068 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-08 18:55:49,068 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-08 18:55:49,069 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d2ad469b39f67aef7d6b850cf65d8a4e in 1272ms, sequenceid=5, compaction requested=false 2023-03-08 18:55:49,069 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:55:49,069 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:49,069 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:49,069 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:55:49,069 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:49,069 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:49,069 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:49,070 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:49,070 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:49,070 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:49,070 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,070 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 18:55:49,070 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 18:55:49,071 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:49,071 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:49,072 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:49,072 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:49,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 18:55:49,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11385afd[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:49,073 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 18:55:49,073 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:49,074 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:49,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:49,074 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 18:55:49,075 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 18:55:49,075 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:49,075 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,075 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:49,077 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 18:55:49,077 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:49,077 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,077 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 18:55:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:49,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 18:55:49,079 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 18:55:49,079 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:49,079 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:49,079 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 18:55:49,079 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:49,080 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,080 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:49,080 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:49,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:49,081 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:49,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:49,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:49,082 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:49,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:49,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,084 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,084 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:49,084 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:49,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:49,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 18:55:49,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:49,084 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:49,084 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:49,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 18:55:49,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:49,085 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:49,085 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:49,084 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:49,085 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,085 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:49,085 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:49,085 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:49,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:49,185 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:49,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:49,187 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:49,187 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:49,187 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:49,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:49,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:49,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 18:55:49,191 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 18:55:49,191 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:49,191 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 18:55:49,191 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 18:55:49,192 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,192 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:49,193 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:49,193 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:49,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:49,193 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:49,193 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:49,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:49,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:49,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:49,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:49,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 18:55:49,194 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 18:55:49,194 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 18:55:49,194 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 18:55:49,194 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:49,194 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 18:55:49,195 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:49,195 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:49,195 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:49,195 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:49,195 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:49,195 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. started... 2023-03-08 18:55:49,195 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. started... 2023-03-08 18:55:49,195 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. started... 2023-03-08 18:55:49,195 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2ad469b39f67aef7d6b850cf65d8a4e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:55:49,195 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8eb59701ce5a41a0784e567680dbb06b 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:55:49,195 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b4625189889702b73dd8f5a775ba90f1 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:55:49,213 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308350d922013b24c97ac985e988419a854 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308350d922013b24c97ac985e988419a854 2023-03-08 18:55:49,611 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308285b5f5f0e7545758a932a0188d59eff to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308285b5f5f0e7545758a932a0188d59eff 2023-03-08 18:55:49,614 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303085ffb64b85c2f41e5b35e81c57c6f68d6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303085ffb64b85c2f41e5b35e81c57c6f68d6 2023-03-08 18:55:49,619 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/8f3adb89023d4b8690741d54a46e9327 2023-03-08 18:55:49,619 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/268e5b504162470fb03bd61ac51f029c 2023-03-08 18:55:50,016 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/f1474ca4f9b84c129c86e46adc13558f 2023-03-08 18:55:50,036 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303083b694c393e49483ab4fd2f3aacb84376 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303083b694c393e49483ab4fd2f3aacb84376 2023-03-08 18:55:50,037 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308fcb4858ebf7e4bb78824f4426d8bcaf1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308fcb4858ebf7e4bb78824f4426d8bcaf1 2023-03-08 18:55:50,429 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303088a9aefaf4ae344d7a6d680a1bb0e2792 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303088a9aefaf4ae344d7a6d680a1bb0e2792 2023-03-08 18:55:50,453 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/00244b1959e447028ff6abcc7af841b4 2023-03-08 18:55:50,456 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/c23d96697ff7471bb214a679d2dae776 2023-03-08 18:55:50,457 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/8f3adb89023d4b8690741d54a46e9327 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/8f3adb89023d4b8690741d54a46e9327 2023-03-08 18:55:50,460 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/268e5b504162470fb03bd61ac51f029c as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/268e5b504162470fb03bd61ac51f029c 2023-03-08 18:55:50,460 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/8f3adb89023d4b8690741d54a46e9327, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 18:55:50,461 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/00244b1959e447028ff6abcc7af841b4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/00244b1959e447028ff6abcc7af841b4 2023-03-08 18:55:50,463 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/268e5b504162470fb03bd61ac51f029c, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 18:55:50,463 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/c23d96697ff7471bb214a679d2dae776 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/c23d96697ff7471bb214a679d2dae776 2023-03-08 18:55:50,465 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/00244b1959e447028ff6abcc7af841b4, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:55:50,465 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b4625189889702b73dd8f5a775ba90f1 in 1270ms, sequenceid=9, compaction requested=false 2023-03-08 18:55:50,465 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:55:50,465 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:50,465 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:50,466 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/c23d96697ff7471bb214a679d2dae776, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:55:50,467 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 8eb59701ce5a41a0784e567680dbb06b in 1272ms, sequenceid=9, compaction requested=false 2023-03-08 18:55:50,467 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:55:50,467 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:50,467 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:50,834 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/7d6e410f976042a8baf7d3030af6401f 2023-03-08 18:55:50,838 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/f1474ca4f9b84c129c86e46adc13558f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/f1474ca4f9b84c129c86e46adc13558f 2023-03-08 18:55:50,841 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/f1474ca4f9b84c129c86e46adc13558f, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 18:55:50,842 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/7d6e410f976042a8baf7d3030af6401f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7d6e410f976042a8baf7d3030af6401f 2023-03-08 18:55:50,845 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7d6e410f976042a8baf7d3030af6401f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:55:50,846 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d2ad469b39f67aef7d6b850cf65d8a4e in 1651ms, sequenceid=9, compaction requested=false 2023-03-08 18:55:50,846 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:50,846 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:50,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:50,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:50,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 18:55:50,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 18:55:50,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,848 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:50,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:50,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:50,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:50,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:50,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:50,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 18:55:50,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6592449b[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:50,850 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 18:55:50,850 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:50,851 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:50,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:50,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:50,852 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 18:55:50,852 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 18:55:50,852 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:50,852 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,852 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:50,856 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 18:55:50,856 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:50,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,856 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 18:55:50,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:50,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:50,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:50,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:50,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:50,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:50,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:50,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:50,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 18:55:50,858 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 18:55:50,858 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:50,858 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:50,858 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 18:55:50,858 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:50,860 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,860 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:50,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:50,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,860 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:50,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:50,860 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:50,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:50,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:50,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:50,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:50,862 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:50,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:50,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:50,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:50,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 18:55:50,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:50,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 18:55:50,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:50,864 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:50,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:50,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:50,864 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:50,965 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:50,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:50,967 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:50,967 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:50,967 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:50,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:50,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:50,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 18:55:50,971 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 18:55:50,971 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:50,971 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 18:55:50,971 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 18:55:50,971 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,971 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:50,973 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,973 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:50,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:50,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:50,974 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:50,974 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:50,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:50,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:50,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:50,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:50,975 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 18:55:50,975 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 18:55:50,975 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 18:55:50,975 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 18:55:50,975 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:50,975 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 18:55:50,975 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:50,975 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. started... 2023-03-08 18:55:50,975 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:50,975 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:50,976 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:50,976 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. started... 2023-03-08 18:55:50,976 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2ad469b39f67aef7d6b850cf65d8a4e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:55:50,976 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b4625189889702b73dd8f5a775ba90f1 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:55:50,976 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:50,976 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. started... 2023-03-08 18:55:50,976 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8eb59701ce5a41a0784e567680dbb06b 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:55:51,388 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ef8551a874c9436c9df74561a680441d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308ef8551a874c9436c9df74561a680441d 2023-03-08 18:55:51,388 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085f8678716061449a987dd8d50ae4ca4c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303085f8678716061449a987dd8d50ae4ca4c 2023-03-08 18:55:51,388 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303081ffbba46468d42679e1f052214175e51 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303081ffbba46468d42679e1f052214175e51 2023-03-08 18:55:51,793 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/d7acb809bb0a4b39b74dbf605f94c21f 2023-03-08 18:55:51,794 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/bdb14a5ab5c24a5cb74a9afa36be00a5 2023-03-08 18:55:51,794 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/135d730485e64227b431943cb9a9d9cb 2023-03-08 18:55:52,210 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d7b43ba043bb40cc94953416025f9e3f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230308d7b43ba043bb40cc94953416025f9e3f 2023-03-08 18:55:52,210 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085c4095e9252c42759ddafa2d227e3224 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303085c4095e9252c42759ddafa2d227e3224 2023-03-08 18:55:52,210 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303089a4a5bfb649b4a0ebb42a779d07d8567 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303089a4a5bfb649b4a0ebb42a779d07d8567 2023-03-08 18:55:52,319 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:55:52,615 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/421138d369ce465bbe228538ba74e73c 2023-03-08 18:55:52,615 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/727c8a464d2443b0b5e26317c2ddf4eb 2023-03-08 18:55:52,616 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/bc29876a6d7041a69dac07f2472f635d 2023-03-08 18:55:52,620 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/d7acb809bb0a4b39b74dbf605f94c21f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/d7acb809bb0a4b39b74dbf605f94c21f 2023-03-08 18:55:52,620 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/135d730485e64227b431943cb9a9d9cb as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/135d730485e64227b431943cb9a9d9cb 2023-03-08 18:55:52,620 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/bdb14a5ab5c24a5cb74a9afa36be00a5 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/bdb14a5ab5c24a5cb74a9afa36be00a5 2023-03-08 18:55:52,623 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/135d730485e64227b431943cb9a9d9cb, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 18:55:52,623 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/d7acb809bb0a4b39b74dbf605f94c21f, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 18:55:52,623 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/bdb14a5ab5c24a5cb74a9afa36be00a5, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 18:55:52,624 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/727c8a464d2443b0b5e26317c2ddf4eb as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/727c8a464d2443b0b5e26317c2ddf4eb 2023-03-08 18:55:52,624 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/bc29876a6d7041a69dac07f2472f635d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/bc29876a6d7041a69dac07f2472f635d 2023-03-08 18:55:52,624 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/421138d369ce465bbe228538ba74e73c as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/421138d369ce465bbe228538ba74e73c 2023-03-08 18:55:52,628 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/bc29876a6d7041a69dac07f2472f635d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:55:52,628 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/727c8a464d2443b0b5e26317c2ddf4eb, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:55:52,628 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/421138d369ce465bbe228538ba74e73c, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:55:52,629 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d2ad469b39f67aef7d6b850cf65d8a4e in 1653ms, sequenceid=13, compaction requested=true 2023-03-08 18:55:52,629 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 8eb59701ce5a41a0784e567680dbb06b in 1653ms, sequenceid=13, compaction requested=true 2023-03-08 18:55:52,629 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:52,629 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:55:52,629 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b4625189889702b73dd8f5a775ba90f1 in 1653ms, sequenceid=13, compaction requested=true 2023-03-08 18:55:52,629 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:52,629 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:55:52,629 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:52,629 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:52,629 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:52,629 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:52,629 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:52,629 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:52,629 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,629 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 18:55:52,629 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 18:55:52,631 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:52,631 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:52,632 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:52,632 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 18:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ea2548[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:52,633 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 18:55:52,633 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:52,634 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:52,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:52,634 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 18:55:52,634 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 18:55:52,634 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:52,634 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,635 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:52,636 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 18:55:52,636 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,636 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:52,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:52,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:52,636 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 18:55:52,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:52,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:52,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:52,638 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 18:55:52,638 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 18:55:52,638 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:52,638 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:52,638 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 18:55:52,638 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:52,644 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,644 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,644 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:52,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:52,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:52,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:52,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:52,645 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:52,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:52,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:52,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:52,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,646 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:52,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:52,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:52,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:52,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:52,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 18:55:52,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:52,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:52,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:52,648 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:52,648 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:52,649 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,649 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:52,649 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:52,649 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,649 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:52,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:52,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:52,649 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:52,749 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:52,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:52,751 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:52,751 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:52,751 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:55:52,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:52,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:52,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 18:55:52,756 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 18:55:52,756 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:52,756 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 18:55:52,756 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 18:55:52,756 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,756 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:52,758 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:52,758 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:52,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:52,758 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:52,758 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:52,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:52,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:52,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:52,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:52,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 18:55:52,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 18:55:52,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 18:55:52,759 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 18:55:52,759 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:52,759 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 18:55:52,759 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:52,759 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:52,759 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:52,759 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:52,759 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:52,759 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. started... 2023-03-08 18:55:52,759 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. started... 2023-03-08 18:55:52,759 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. started... 2023-03-08 18:55:52,760 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8eb59701ce5a41a0784e567680dbb06b 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:55:52,760 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b4625189889702b73dd8f5a775ba90f1 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:55:52,760 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2ad469b39f67aef7d6b850cf65d8a4e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:55:53,174 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030897949acc7a3949429007ffdc12960a6e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023030897949acc7a3949429007ffdc12960a6e 2023-03-08 18:55:53,174 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030872799619d0e241aba477c34ae3b1a2de to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023030872799619d0e241aba477c34ae3b1a2de 2023-03-08 18:55:53,174 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d7a955767cb04ad3b8f31148212d8f6c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308d7a955767cb04ad3b8f31148212d8f6c 2023-03-08 18:55:53,579 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/221a308a9eb446a398b714825daa38d6 2023-03-08 18:55:53,579 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/9c5a9036aaa547e085ccab72f73e1fed 2023-03-08 18:55:53,579 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/14b9f399bc314d63ab9aa3c3f0941085 2023-03-08 18:55:54,001 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030886b431ec60ff47baaab9cd08717c8065 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030886b431ec60ff47baaab9cd08717c8065 2023-03-08 18:55:54,002 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308053c53b86fdf440cbb2d66fb0370007e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308053c53b86fdf440cbb2d66fb0370007e 2023-03-08 18:55:54,002 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303082039bd764ab249f1b6f6c6729ccebb31 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303082039bd764ab249f1b6f6c6729ccebb31 2023-03-08 18:55:54,007 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/209f4982db94490a9bc2a7d06558f52e 2023-03-08 18:55:54,010 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/221a308a9eb446a398b714825daa38d6 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/221a308a9eb446a398b714825daa38d6 2023-03-08 18:55:54,014 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/221a308a9eb446a398b714825daa38d6, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 18:55:54,014 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/209f4982db94490a9bc2a7d06558f52e as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/209f4982db94490a9bc2a7d06558f52e 2023-03-08 18:55:54,018 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/209f4982db94490a9bc2a7d06558f52e, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:55:54,018 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 8eb59701ce5a41a0784e567680dbb06b in 1258ms, sequenceid=17, compaction requested=true 2023-03-08 18:55:54,019 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:55:54,019 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:54,019 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:54,406 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/65fc9ac8406541dbb8fdb38149485b83 2023-03-08 18:55:54,407 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/7c5b876edad543beacb6e42fa670f25f 2023-03-08 18:55:54,410 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/14b9f399bc314d63ab9aa3c3f0941085 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/14b9f399bc314d63ab9aa3c3f0941085 2023-03-08 18:55:54,411 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/9c5a9036aaa547e085ccab72f73e1fed as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/9c5a9036aaa547e085ccab72f73e1fed 2023-03-08 18:55:54,414 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/14b9f399bc314d63ab9aa3c3f0941085, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 18:55:54,414 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/9c5a9036aaa547e085ccab72f73e1fed, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 18:55:54,415 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/65fc9ac8406541dbb8fdb38149485b83 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/65fc9ac8406541dbb8fdb38149485b83 2023-03-08 18:55:54,415 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/7c5b876edad543beacb6e42fa670f25f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7c5b876edad543beacb6e42fa670f25f 2023-03-08 18:55:54,419 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7c5b876edad543beacb6e42fa670f25f, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:55:54,419 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/65fc9ac8406541dbb8fdb38149485b83, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:55:54,420 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d2ad469b39f67aef7d6b850cf65d8a4e in 1660ms, sequenceid=17, compaction requested=true 2023-03-08 18:55:54,420 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b4625189889702b73dd8f5a775ba90f1 in 1660ms, sequenceid=17, compaction requested=true 2023-03-08 18:55:54,420 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:54,420 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:54,420 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:55:54,420 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:54,420 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:54,420 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:54,420 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:54,420 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,420 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 18:55:54,420 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 18:55:54,422 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:54,422 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:54,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:54,422 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:54,422 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:54,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:54,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:54,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:54,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:54,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 18:55:54,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d69e8af[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:54,424 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 18:55:54,424 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:54,426 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:54,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:54,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:54,426 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 18:55:54,426 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 18:55:54,426 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,426 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:54,426 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:54,428 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 18:55:54,428 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,428 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:54,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:54,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:54,428 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 18:55:54,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:54,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:54,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:54,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:54,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:54,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:54,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 18:55:54,430 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 18:55:54,430 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:54,430 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:54,431 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 18:55:54,431 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:54,432 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,432 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:54,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,432 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:54,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:54,433 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,434 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:54,437 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:54,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:54,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:54,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,437 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:54,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:54,438 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:54,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:54,538 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:54,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:54,549 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:54,557 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:54,566 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:55:54,567 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:55:54,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:55:54,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 18:55:54,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 18:55:54,595 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 18:55:54,595 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:54,595 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 18:55:54,595 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 18:55:54,595 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,595 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:54,598 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:54,598 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:54,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:54,598 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:54,598 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:55:54,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:54,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:54,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:55:54,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:54,599 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 18:55:54,599 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 18:55:54,599 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 18:55:54,599 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 18:55:54,599 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:55:54,599 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 18:55:54,599 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:54,599 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. started... 2023-03-08 18:55:54,599 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:54,599 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:55:54,600 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:54,600 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2ad469b39f67aef7d6b850cf65d8a4e 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:55:54,600 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. started... 2023-03-08 18:55:54,600 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. started... 2023-03-08 18:55:54,600 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8eb59701ce5a41a0784e567680dbb06b 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:55:54,600 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:55:54,600 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b4625189889702b73dd8f5a775ba90f1 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:55:55,024 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/61b9c20d5756412a83e2c8df7d922def 2023-03-08 18:55:55,026 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/85aeb0a7a376481bb80764736669b4ad 2023-03-08 18:55:55,026 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/22af31a652664eae8744e0401e5cf2f1 2023-03-08 18:55:55,028 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 61b9c20d5756412a83e2c8df7d922def 2023-03-08 18:55:55,030 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 85aeb0a7a376481bb80764736669b4ad 2023-03-08 18:55:55,030 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22af31a652664eae8744e0401e5cf2f1 2023-03-08 18:55:55,454 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/f640216cf18f41ad83b7d0f6efa42c11 2023-03-08 18:55:55,458 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f640216cf18f41ad83b7d0f6efa42c11 2023-03-08 18:55:55,458 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/22af31a652664eae8744e0401e5cf2f1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/22af31a652664eae8744e0401e5cf2f1 2023-03-08 18:55:55,461 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22af31a652664eae8744e0401e5cf2f1 2023-03-08 18:55:55,461 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/22af31a652664eae8744e0401e5cf2f1, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 18:55:55,462 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/f640216cf18f41ad83b7d0f6efa42c11 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/f640216cf18f41ad83b7d0f6efa42c11 2023-03-08 18:55:55,465 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f640216cf18f41ad83b7d0f6efa42c11 2023-03-08 18:55:55,465 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/f640216cf18f41ad83b7d0f6efa42c11, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 18:55:55,466 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 8eb59701ce5a41a0784e567680dbb06b in 866ms, sequenceid=23, compaction requested=true 2023-03-08 18:55:55,466 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:55:55,466 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:55,466 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:55:55,853 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/79c28b7e7ee741b7958f30ea8d7094a7 2023-03-08 18:55:55,853 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/235b2bb8df6c4419bdf8f14c15673c9d 2023-03-08 18:55:55,856 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 79c28b7e7ee741b7958f30ea8d7094a7 2023-03-08 18:55:55,856 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 235b2bb8df6c4419bdf8f14c15673c9d 2023-03-08 18:55:55,857 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/61b9c20d5756412a83e2c8df7d922def as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/61b9c20d5756412a83e2c8df7d922def 2023-03-08 18:55:55,857 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/85aeb0a7a376481bb80764736669b4ad as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/85aeb0a7a376481bb80764736669b4ad 2023-03-08 18:55:55,860 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 61b9c20d5756412a83e2c8df7d922def 2023-03-08 18:55:55,860 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/61b9c20d5756412a83e2c8df7d922def, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 18:55:55,860 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 85aeb0a7a376481bb80764736669b4ad 2023-03-08 18:55:55,860 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/85aeb0a7a376481bb80764736669b4ad, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 18:55:55,861 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/79c28b7e7ee741b7958f30ea8d7094a7 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/79c28b7e7ee741b7958f30ea8d7094a7 2023-03-08 18:55:55,861 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/235b2bb8df6c4419bdf8f14c15673c9d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/235b2bb8df6c4419bdf8f14c15673c9d 2023-03-08 18:55:55,864 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 79c28b7e7ee741b7958f30ea8d7094a7 2023-03-08 18:55:55,864 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/79c28b7e7ee741b7958f30ea8d7094a7, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 18:55:55,865 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 235b2bb8df6c4419bdf8f14c15673c9d 2023-03-08 18:55:55,865 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/235b2bb8df6c4419bdf8f14c15673c9d, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 18:55:55,865 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d2ad469b39f67aef7d6b850cf65d8a4e in 1265ms, sequenceid=23, compaction requested=true 2023-03-08 18:55:55,865 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:55,865 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:55,865 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:55:55,865 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b4625189889702b73dd8f5a775ba90f1 in 1265ms, sequenceid=23, compaction requested=true 2023-03-08 18:55:55,865 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:55:55,865 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:55:55,865 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:55:55,865 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:55:55,865 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,866 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 18:55:55,866 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 18:55:55,867 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:55,867 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:55,867 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:55,867 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:55:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:55,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 18:55:55,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b06359f[Count = 0] remaining members to acquire global barrier 2023-03-08 18:55:55,868 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 18:55:55,869 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:55,870 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:55,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:55,870 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 18:55:55,870 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 18:55:55,870 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,870 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:55:55,870 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:55:55,871 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,871 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 18:55:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:55,871 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:55,871 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 18:55:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:55,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:55,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:55:55,873 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 18:55:55,873 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 18:55:55,874 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:55:55,874 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:55:55,874 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 18:55:55,874 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:55:55,883 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:55,883 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:55,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:55,883 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:55,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:55:55,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:55:55,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:55,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:55,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:55:55,884 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:55,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:55,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:55,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:55,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:55:55,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:55,885 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:55:55,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 18:55:55,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:55,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 18:55:55,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 18:55:55,887 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:55:55,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:55:55,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:55:55,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 18:55:55,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:55:55,888 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:55:55,887 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 18:55:55,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:55:55,888 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 18:55:55,988 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:55:55,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:55:55,989 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:55,989 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): d2ad469b39f67aef7d6b850cf65d8a4e/family1 is initiating major compaction (all files) 2023-03-08 18:55:55,989 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:55,989 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:55,990 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of d2ad469b39f67aef7d6b850cf65d8a4e/family1 in ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:55,990 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/6aeec20bf44542a782ed2d04b28ea7a1, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/f1474ca4f9b84c129c86e46adc13558f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/bdb14a5ab5c24a5cb74a9afa36be00a5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/9c5a9036aaa547e085ccab72f73e1fed, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/61b9c20d5756412a83e2c8df7d922def] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp, totalSize=26.6 K 2023-03-08 18:55:55,990 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 6aeec20bf44542a782ed2d04b28ea7a1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301747787 2023-03-08 18:55:55,991 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting f1474ca4f9b84c129c86e46adc13558f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301749187 2023-03-08 18:55:55,991 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting bdb14a5ab5c24a5cb74a9afa36be00a5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301750967 2023-03-08 18:55:55,991 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 9c5a9036aaa547e085ccab72f73e1fed, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301752751 2023-03-08 18:55:55,992 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 61b9c20d5756412a83e2c8df7d922def, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:55:56,001 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): d2ad469b39f67aef7d6b850cf65d8a4e#family1#compaction#150 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:56,409 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308c67485f9685242b284a2d96839b56c09_del 2023-03-08 18:55:56,409 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c67485f9685242b284a2d96839b56c09_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308c67485f9685242b284a2d96839b56c09_del 2023-03-08 18:55:57,224 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d246041a909348839ae356166a2b0b25 2023-03-08 18:55:57,225 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family1/d246041a909348839ae356166a2b0b25 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/d246041a909348839ae356166a2b0b25 2023-03-08 18:55:57,229 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d246041a909348839ae356166a2b0b25 2023-03-08 18:55:57,229 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d2ad469b39f67aef7d6b850cf65d8a4e/family1 of d2ad469b39f67aef7d6b850cf65d8a4e into d246041a909348839ae356166a2b0b25(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:57,230 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:57,230 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:57,230 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): d2ad469b39f67aef7d6b850cf65d8a4e/family2 is initiating major compaction (all files) 2023-03-08 18:55:57,230 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:57,230 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:57,230 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of d2ad469b39f67aef7d6b850cf65d8a4e/family2 in ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:55:57,230 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/29edb21579114693b285fe61de1810aa, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7d6e410f976042a8baf7d3030af6401f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/bc29876a6d7041a69dac07f2472f635d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7c5b876edad543beacb6e42fa670f25f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/79c28b7e7ee741b7958f30ea8d7094a7] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp, totalSize=25.3 K 2023-03-08 18:55:57,231 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 29edb21579114693b285fe61de1810aa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301747787 2023-03-08 18:55:57,231 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 7d6e410f976042a8baf7d3030af6401f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301749187 2023-03-08 18:55:57,231 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting bc29876a6d7041a69dac07f2472f635d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301750967 2023-03-08 18:55:57,231 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 7c5b876edad543beacb6e42fa670f25f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301752751 2023-03-08 18:55:57,232 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 79c28b7e7ee741b7958f30ea8d7094a7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:55:57,240 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): d2ad469b39f67aef7d6b850cf65d8a4e#family2#compaction#151 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:57,254 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303083f2c514d39f34293881a57b8a5d0f18f_del 2023-03-08 18:55:57,254 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303083f2c514d39f34293881a57b8a5d0f18f_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303083f2c514d39f34293881a57b8a5d0f18f_del 2023-03-08 18:55:57,281 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/6aeec20bf44542a782ed2d04b28ea7a1, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/f1474ca4f9b84c129c86e46adc13558f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/bdb14a5ab5c24a5cb74a9afa36be00a5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/9c5a9036aaa547e085ccab72f73e1fed, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/61b9c20d5756412a83e2c8df7d922def] to archive 2023-03-08 18:55:57,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:57,284 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/6aeec20bf44542a782ed2d04b28ea7a1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/6aeec20bf44542a782ed2d04b28ea7a1 2023-03-08 18:55:57,285 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/f1474ca4f9b84c129c86e46adc13558f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/f1474ca4f9b84c129c86e46adc13558f 2023-03-08 18:55:57,286 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/bdb14a5ab5c24a5cb74a9afa36be00a5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/bdb14a5ab5c24a5cb74a9afa36be00a5 2023-03-08 18:55:57,287 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/9c5a9036aaa547e085ccab72f73e1fed to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/9c5a9036aaa547e085ccab72f73e1fed 2023-03-08 18:55:57,288 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/61b9c20d5756412a83e2c8df7d922def to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/61b9c20d5756412a83e2c8df7d922def 2023-03-08 18:55:58,067 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a9737493d2a43bb8dd7fb23a0e066c0 2023-03-08 18:55:58,068 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/.tmp/family2/7a9737493d2a43bb8dd7fb23a0e066c0 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7a9737493d2a43bb8dd7fb23a0e066c0 2023-03-08 18:55:58,071 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a9737493d2a43bb8dd7fb23a0e066c0 2023-03-08 18:55:58,072 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d2ad469b39f67aef7d6b850cf65d8a4e/family2 of d2ad469b39f67aef7d6b850cf65d8a4e into 7a9737493d2a43bb8dd7fb23a0e066c0(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:55:58,072 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:55:58,072 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:58,072 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 8eb59701ce5a41a0784e567680dbb06b/family1 is initiating major compaction (all files) 2023-03-08 18:55:58,072 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:58,072 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:58,073 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 8eb59701ce5a41a0784e567680dbb06b/family1 in ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:58,073 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f83428eea1624a94b0ed90565aafc027, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/268e5b504162470fb03bd61ac51f029c, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/d7acb809bb0a4b39b74dbf605f94c21f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/221a308a9eb446a398b714825daa38d6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/22af31a652664eae8744e0401e5cf2f1] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp, totalSize=26.6 K 2023-03-08 18:55:58,074 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting f83428eea1624a94b0ed90565aafc027, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301747788 2023-03-08 18:55:58,074 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 268e5b504162470fb03bd61ac51f029c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301749187 2023-03-08 18:55:58,074 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting d7acb809bb0a4b39b74dbf605f94c21f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301750967 2023-03-08 18:55:58,075 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 221a308a9eb446a398b714825daa38d6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301752751 2023-03-08 18:55:58,075 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 22af31a652664eae8744e0401e5cf2f1, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:55:58,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/29edb21579114693b285fe61de1810aa, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7d6e410f976042a8baf7d3030af6401f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/bc29876a6d7041a69dac07f2472f635d, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7c5b876edad543beacb6e42fa670f25f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/79c28b7e7ee741b7958f30ea8d7094a7] to archive 2023-03-08 18:55:58,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:58,082 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 8eb59701ce5a41a0784e567680dbb06b#family1#compaction#152 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:58,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/29edb21579114693b285fe61de1810aa to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/29edb21579114693b285fe61de1810aa 2023-03-08 18:55:58,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7d6e410f976042a8baf7d3030af6401f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7d6e410f976042a8baf7d3030af6401f 2023-03-08 18:55:58,085 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/bc29876a6d7041a69dac07f2472f635d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/bc29876a6d7041a69dac07f2472f635d 2023-03-08 18:55:58,087 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7c5b876edad543beacb6e42fa670f25f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/7c5b876edad543beacb6e42fa670f25f 2023-03-08 18:55:58,088 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/79c28b7e7ee741b7958f30ea8d7094a7 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family2/79c28b7e7ee741b7958f30ea8d7094a7 2023-03-08 18:55:58,490 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308440fd8d57bce496887fc1d0a5598d182_del 2023-03-08 18:55:58,490 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308440fd8d57bce496887fc1d0a5598d182_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308440fd8d57bce496887fc1d0a5598d182_del 2023-03-08 18:55:59,304 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cf93ea3f3033491cad06f8f8afc0ca46 2023-03-08 18:55:59,304 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family1/cf93ea3f3033491cad06f8f8afc0ca46 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/cf93ea3f3033491cad06f8f8afc0ca46 2023-03-08 18:55:59,307 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cf93ea3f3033491cad06f8f8afc0ca46 2023-03-08 18:55:59,308 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8eb59701ce5a41a0784e567680dbb06b/family1 of 8eb59701ce5a41a0784e567680dbb06b into cf93ea3f3033491cad06f8f8afc0ca46(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:55:59,308 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:55:59,308 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:55:59,309 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 8eb59701ce5a41a0784e567680dbb06b/family2 is initiating major compaction (all files) 2023-03-08 18:55:59,309 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:55:59,309 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:55:59,309 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 8eb59701ce5a41a0784e567680dbb06b/family2 in ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:55:59,309 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/6835c1db1860462dae97ca0d022af34b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/c23d96697ff7471bb214a679d2dae776, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/421138d369ce465bbe228538ba74e73c, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/209f4982db94490a9bc2a7d06558f52e, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/f640216cf18f41ad83b7d0f6efa42c11] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp, totalSize=25.3 K 2023-03-08 18:55:59,309 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 6835c1db1860462dae97ca0d022af34b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301747788 2023-03-08 18:55:59,310 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting c23d96697ff7471bb214a679d2dae776, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301749187 2023-03-08 18:55:59,310 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 421138d369ce465bbe228538ba74e73c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301750967 2023-03-08 18:55:59,310 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 209f4982db94490a9bc2a7d06558f52e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301752751 2023-03-08 18:55:59,310 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting f640216cf18f41ad83b7d0f6efa42c11, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:55:59,317 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 8eb59701ce5a41a0784e567680dbb06b#family2#compaction#153 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:55:59,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f83428eea1624a94b0ed90565aafc027, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/268e5b504162470fb03bd61ac51f029c, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/d7acb809bb0a4b39b74dbf605f94c21f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/221a308a9eb446a398b714825daa38d6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/22af31a652664eae8744e0401e5cf2f1] to archive 2023-03-08 18:55:59,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:55:59,388 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f83428eea1624a94b0ed90565aafc027 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f83428eea1624a94b0ed90565aafc027 2023-03-08 18:55:59,389 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/268e5b504162470fb03bd61ac51f029c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/268e5b504162470fb03bd61ac51f029c 2023-03-08 18:55:59,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/d7acb809bb0a4b39b74dbf605f94c21f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/d7acb809bb0a4b39b74dbf605f94c21f 2023-03-08 18:55:59,391 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/221a308a9eb446a398b714825daa38d6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/221a308a9eb446a398b714825daa38d6 2023-03-08 18:55:59,391 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/22af31a652664eae8744e0401e5cf2f1 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/22af31a652664eae8744e0401e5cf2f1 2023-03-08 18:55:59,724 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030813d10d4120ad48758d103dc2e4494c6b_del 2023-03-08 18:55:59,725 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030813d10d4120ad48758d103dc2e4494c6b_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023030813d10d4120ad48758d103dc2e4494c6b_del 2023-03-08 18:56:00,537 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c3c24205ef24e49919162423a3b4fda 2023-03-08 18:56:00,538 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/.tmp/family2/2c3c24205ef24e49919162423a3b4fda as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/2c3c24205ef24e49919162423a3b4fda 2023-03-08 18:56:00,542 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c3c24205ef24e49919162423a3b4fda 2023-03-08 18:56:00,543 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8eb59701ce5a41a0784e567680dbb06b/family2 of 8eb59701ce5a41a0784e567680dbb06b into 2c3c24205ef24e49919162423a3b4fda(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:56:00,543 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:56:00,543 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:56:00,543 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): b4625189889702b73dd8f5a775ba90f1/family1 is initiating major compaction (all files) 2023-03-08 18:56:00,543 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:56:00,543 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:56:00,543 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of b4625189889702b73dd8f5a775ba90f1/family1 in ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:56:00,543 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7f8309375eb14472adf0d6ee1c31d6d4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/8f3adb89023d4b8690741d54a46e9327, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/135d730485e64227b431943cb9a9d9cb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/14b9f399bc314d63ab9aa3c3f0941085, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/85aeb0a7a376481bb80764736669b4ad] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp, totalSize=26.6 K 2023-03-08 18:56:00,544 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 7f8309375eb14472adf0d6ee1c31d6d4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301747787 2023-03-08 18:56:00,544 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 8f3adb89023d4b8690741d54a46e9327, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301749187 2023-03-08 18:56:00,544 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 135d730485e64227b431943cb9a9d9cb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301750967 2023-03-08 18:56:00,544 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 14b9f399bc314d63ab9aa3c3f0941085, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301752751 2023-03-08 18:56:00,545 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 85aeb0a7a376481bb80764736669b4ad, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:56:00,552 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): b4625189889702b73dd8f5a775ba90f1#family1#compaction#154 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:00,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/6835c1db1860462dae97ca0d022af34b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/c23d96697ff7471bb214a679d2dae776, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/421138d369ce465bbe228538ba74e73c, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/209f4982db94490a9bc2a7d06558f52e, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/f640216cf18f41ad83b7d0f6efa42c11] to archive 2023-03-08 18:56:00,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:00,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/6835c1db1860462dae97ca0d022af34b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/6835c1db1860462dae97ca0d022af34b 2023-03-08 18:56:00,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/c23d96697ff7471bb214a679d2dae776 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/c23d96697ff7471bb214a679d2dae776 2023-03-08 18:56:00,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/421138d369ce465bbe228538ba74e73c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/421138d369ce465bbe228538ba74e73c 2023-03-08 18:56:00,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/209f4982db94490a9bc2a7d06558f52e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/209f4982db94490a9bc2a7d06558f52e 2023-03-08 18:56:00,588 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/f640216cf18f41ad83b7d0f6efa42c11 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family2/f640216cf18f41ad83b7d0f6efa42c11 2023-03-08 18:56:00,963 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030847287a3aa600436eb2b446b783f65c37_del 2023-03-08 18:56:00,963 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030847287a3aa600436eb2b446b783f65c37_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023030847287a3aa600436eb2b446b783f65c37_del 2023-03-08 18:56:01,776 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6caac7153e9e41348f87b1e5d5aa159f 2023-03-08 18:56:01,776 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family1/6caac7153e9e41348f87b1e5d5aa159f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/6caac7153e9e41348f87b1e5d5aa159f 2023-03-08 18:56:01,780 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6caac7153e9e41348f87b1e5d5aa159f 2023-03-08 18:56:01,781 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b4625189889702b73dd8f5a775ba90f1/family1 of b4625189889702b73dd8f5a775ba90f1 into 6caac7153e9e41348f87b1e5d5aa159f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:56:01,781 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:56:01,781 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:56:01,781 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): b4625189889702b73dd8f5a775ba90f1/family2 is initiating major compaction (all files) 2023-03-08 18:56:01,782 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:56:01,782 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:56:01,782 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of b4625189889702b73dd8f5a775ba90f1/family2 in ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:56:01,782 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/ea67bef51fc14a3d9278d9ca60ddfbff, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/00244b1959e447028ff6abcc7af841b4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/727c8a464d2443b0b5e26317c2ddf4eb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/65fc9ac8406541dbb8fdb38149485b83, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/235b2bb8df6c4419bdf8f14c15673c9d] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp, totalSize=25.3 K 2023-03-08 18:56:01,782 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting ea67bef51fc14a3d9278d9ca60ddfbff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301747787 2023-03-08 18:56:01,783 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 00244b1959e447028ff6abcc7af841b4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301749187 2023-03-08 18:56:01,783 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 727c8a464d2443b0b5e26317c2ddf4eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301750967 2023-03-08 18:56:01,783 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 65fc9ac8406541dbb8fdb38149485b83, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301752751 2023-03-08 18:56:01,784 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 235b2bb8df6c4419bdf8f14c15673c9d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:56:01,790 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7f8309375eb14472adf0d6ee1c31d6d4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/8f3adb89023d4b8690741d54a46e9327, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/135d730485e64227b431943cb9a9d9cb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/14b9f399bc314d63ab9aa3c3f0941085, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/85aeb0a7a376481bb80764736669b4ad] to archive 2023-03-08 18:56:01,790 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:01,791 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): b4625189889702b73dd8f5a775ba90f1#family2#compaction#155 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:01,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7f8309375eb14472adf0d6ee1c31d6d4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7f8309375eb14472adf0d6ee1c31d6d4 2023-03-08 18:56:01,793 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/8f3adb89023d4b8690741d54a46e9327 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/8f3adb89023d4b8690741d54a46e9327 2023-03-08 18:56:01,794 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/135d730485e64227b431943cb9a9d9cb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/135d730485e64227b431943cb9a9d9cb 2023-03-08 18:56:01,796 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/14b9f399bc314d63ab9aa3c3f0941085 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/14b9f399bc314d63ab9aa3c3f0941085 2023-03-08 18:56:01,797 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/85aeb0a7a376481bb80764736669b4ad to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/85aeb0a7a376481bb80764736669b4ad 2023-03-08 18:56:02,201 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303085773b4cea3094e46b85ebf9cde891c7a_del 2023-03-08 18:56:02,201 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303085773b4cea3094e46b85ebf9cde891c7a_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303085773b4cea3094e46b85ebf9cde891c7a_del 2023-03-08 18:56:03,016 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac39568b754d48f18c0109d1a46d69f2 2023-03-08 18:56:03,017 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/.tmp/family2/ac39568b754d48f18c0109d1a46d69f2 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/ac39568b754d48f18c0109d1a46d69f2 2023-03-08 18:56:03,021 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac39568b754d48f18c0109d1a46d69f2 2023-03-08 18:56:03,022 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b4625189889702b73dd8f5a775ba90f1/family2 of b4625189889702b73dd8f5a775ba90f1 into ac39568b754d48f18c0109d1a46d69f2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:56:03,022 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:56:03,025 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:03,027 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:03,031 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:03,032 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:03,037 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 18:56:03,051 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-08 18:56:03,054 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308c67485f9685242b284a2d96839b56c09_del 2023-03-08 18:56:03,057 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030847287a3aa600436eb2b446b783f65c37_del 2023-03-08 18:56:03,060 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308440fd8d57bce496887fc1d0a5598d182_del 2023-03-08 18:56:03,060 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-08 18:56:03,061 DEBUG [Listener at localhost/36281] zookeeper.ReadOnlyZKClient(139): Connect 0x20d7c20f to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:56:03,066 DEBUG [Listener at localhost/36281] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d841d14, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:56:03,067 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 18:56:03,068 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 18:56:03,067 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 18:56:03,088 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/ea67bef51fc14a3d9278d9ca60ddfbff, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/00244b1959e447028ff6abcc7af841b4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/727c8a464d2443b0b5e26317c2ddf4eb, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/65fc9ac8406541dbb8fdb38149485b83, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/235b2bb8df6c4419bdf8f14c15673c9d] to archive 2023-03-08 18:56:03,088 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:03,090 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/ea67bef51fc14a3d9278d9ca60ddfbff to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/ea67bef51fc14a3d9278d9ca60ddfbff 2023-03-08 18:56:03,091 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/00244b1959e447028ff6abcc7af841b4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/00244b1959e447028ff6abcc7af841b4 2023-03-08 18:56:03,092 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303088382c604d6654d6eb485caa6e9b054d4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303088382c604d6654d6eb485caa6e9b054d4 2023-03-08 18:56:03,093 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/727c8a464d2443b0b5e26317c2ddf4eb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/727c8a464d2443b0b5e26317c2ddf4eb 2023-03-08 18:56:03,093 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 18:56:03,094 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/65fc9ac8406541dbb8fdb38149485b83 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/65fc9ac8406541dbb8fdb38149485b83 2023-03-08 18:56:03,095 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/235b2bb8df6c4419bdf8f14c15673c9d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family2/235b2bb8df6c4419bdf8f14c15673c9d 2023-03-08 18:56:03,096 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308221b8cf19d2d4bd0b6eb0cf0c491871c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308221b8cf19d2d4bd0b6eb0cf0c491871c 2023-03-08 18:56:03,097 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 18:56:03,107 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:56:03,122 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:56:03,122 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c8ddbb5 to 127.0.0.1:54098 2023-03-08 18:56:03,122 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:03,123 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:56:03,123 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x155ddbd8 to 127.0.0.1:54098 2023-03-08 18:56:03,123 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:03,123 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:34276, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:56:03,138 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:56:03,141 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:43978, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:56:03,159 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/79937b500f6c4f68838d625579b4207b first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-08 18:56:03,159 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/26b9748028c740888cfda3ddbd629074 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-08 18:56:03,160 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/26b9748028c740888cfda3ddbd629074}] 2023-03-08 18:56:03,160 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/79937b500f6c4f68838d625579b4207b}] 2023-03-08 18:56:03,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/26b9748028c740888cfda3ddbd629074 for inclusion in b4625189889702b73dd8f5a775ba90f1/family1 2023-03-08 18:56:03,168 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/79937b500f6c4f68838d625579b4207b for inclusion in 8eb59701ce5a41a0784e567680dbb06b/family1 2023-03-08 18:56:03,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-08 18:56:03,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 18:56:03,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-08 18:56:03,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 18:56:03,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:56:03,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:56:03,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/79937b500f6c4f68838d625579b4207b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__nvgtn5ma6tupgqm3ida862smqdnmapsl9ehocidiu5k10ru0s1fmuads62pga1km/family1/79937b500f6c4f68838d625579b4207b 2023-03-08 18:56:03,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/26b9748028c740888cfda3ddbd629074 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__9pnvhb3oeck1motaunpqaana3h1tkovpk630300qeu2u4691g6kojjtbr6g0r26s/family1/26b9748028c740888cfda3ddbd629074 2023-03-08 18:56:03,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__9pnvhb3oeck1motaunpqaana3h1tkovpk630300qeu2u4691g6kojjtbr6g0r26s/family1/26b9748028c740888cfda3ddbd629074 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7dac165b8c404b099ad30ff4b0e0162e_SeqId_28_ 2023-03-08 18:56:03,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__nvgtn5ma6tupgqm3ida862smqdnmapsl9ehocidiu5k10ru0s1fmuads62pga1km/family1/79937b500f6c4f68838d625579b4207b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f05e822a93024ed8a9d0c7ad8e370107_SeqId_28_ 2023-03-08 18:56:03,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__9pnvhb3oeck1motaunpqaana3h1tkovpk630300qeu2u4691g6kojjtbr6g0r26s/family1/26b9748028c740888cfda3ddbd629074 into b4625189889702b73dd8f5a775ba90f1/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7dac165b8c404b099ad30ff4b0e0162e_SeqId_28_ - updating store file list. 2023-03-08 18:56:03,176 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__nvgtn5ma6tupgqm3ida862smqdnmapsl9ehocidiu5k10ru0s1fmuads62pga1km/family1/79937b500f6c4f68838d625579b4207b into 8eb59701ce5a41a0784e567680dbb06b/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f05e822a93024ed8a9d0c7ad8e370107_SeqId_28_ - updating store file list. 2023-03-08 18:56:03,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7dac165b8c404b099ad30ff4b0e0162e_SeqId_28_ into b4625189889702b73dd8f5a775ba90f1/family1 2023-03-08 18:56:03,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__9pnvhb3oeck1motaunpqaana3h1tkovpk630300qeu2u4691g6kojjtbr6g0r26s/family1/26b9748028c740888cfda3ddbd629074 into b4625189889702b73dd8f5a775ba90f1/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/family1/7dac165b8c404b099ad30ff4b0e0162e_SeqId_28_) 2023-03-08 18:56:03,180 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f05e822a93024ed8a9d0c7ad8e370107_SeqId_28_ into 8eb59701ce5a41a0784e567680dbb06b/family1 2023-03-08 18:56:03,180 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__nvgtn5ma6tupgqm3ida862smqdnmapsl9ehocidiu5k10ru0s1fmuads62pga1km/family1/79937b500f6c4f68838d625579b4207b into 8eb59701ce5a41a0784e567680dbb06b/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/family1/f05e822a93024ed8a9d0c7ad8e370107_SeqId_28_) 2023-03-08 18:56:03,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__9pnvhb3oeck1motaunpqaana3h1tkovpk630300qeu2u4691g6kojjtbr6g0r26s/family1/26b9748028c740888cfda3ddbd629074 2023-03-08 18:56:03,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__nvgtn5ma6tupgqm3ida862smqdnmapsl9ehocidiu5k10ru0s1fmuads62pga1km/family1/79937b500f6c4f68838d625579b4207b 2023-03-08 18:56:03,183 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 18:56:03,183 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 18:56:03,183 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:03,183 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:03,185 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ca4e82da78564c4c9b881c08bcecdd79 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ca4e82da78564c4c9b881c08bcecdd79 2023-03-08 18:56:03,185 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 18:56:03,185 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 18:56:03,185 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303084d0870d0a6c64d388c89afb4aceda2c2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303084d0870d0a6c64d388c89afb4aceda2c2 2023-03-08 18:56:03,185 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 18:56:03,185 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 18:56:03,185 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 18:56:03,185 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 18:56:03,892 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c324d6d9fbc34f0aaddb6bd58240ef25 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308c324d6d9fbc34f0aaddb6bd58240ef25 2023-03-08 18:56:03,893 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 18:56:03,921 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/24e4ea64ddc647278d38bab6b34eb466 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-08 18:56:03,922 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/24e4ea64ddc647278d38bab6b34eb466}] 2023-03-08 18:56:03,923 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/24e4ea64ddc647278d38bab6b34eb466 for inclusion in d2ad469b39f67aef7d6b850cf65d8a4e/family1 2023-03-08 18:56:03,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-08 18:56:03,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 18:56:03,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:56:03,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/24e4ea64ddc647278d38bab6b34eb466 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__6lb6kotlvkc34cf6ln7q800rca4ris041eqqr2jm348naduivj8ur14k6hiohlrv/family1/24e4ea64ddc647278d38bab6b34eb466 2023-03-08 18:56:03,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__6lb6kotlvkc34cf6ln7q800rca4ris041eqqr2jm348naduivj8ur14k6hiohlrv/family1/24e4ea64ddc647278d38bab6b34eb466 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/e744583ce5064f999338c0a5a41d3b52_SeqId_28_ 2023-03-08 18:56:03,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__6lb6kotlvkc34cf6ln7q800rca4ris041eqqr2jm348naduivj8ur14k6hiohlrv/family1/24e4ea64ddc647278d38bab6b34eb466 into d2ad469b39f67aef7d6b850cf65d8a4e/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/e744583ce5064f999338c0a5a41d3b52_SeqId_28_ - updating store file list. 2023-03-08 18:56:03,930 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/e744583ce5064f999338c0a5a41d3b52_SeqId_28_ into d2ad469b39f67aef7d6b850cf65d8a4e/family1 2023-03-08 18:56:03,930 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__6lb6kotlvkc34cf6ln7q800rca4ris041eqqr2jm348naduivj8ur14k6hiohlrv/family1/24e4ea64ddc647278d38bab6b34eb466 into d2ad469b39f67aef7d6b850cf65d8a4e/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/family1/e744583ce5064f999338c0a5a41d3b52_SeqId_28_) 2023-03-08 18:56:03,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__ns_testMinorCompaction__6lb6kotlvkc34cf6ln7q800rca4ris041eqqr2jm348naduivj8ur14k6hiohlrv/family1/24e4ea64ddc647278d38bab6b34eb466 2023-03-08 18:56:03,932 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 18:56:03,933 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:03,934 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308d221c789f1bf41feba96deaa9606f6f2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308d221c789f1bf41feba96deaa9606f6f2 2023-03-08 18:56:03,934 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 18:56:03,934 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 18:56:03,934 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 18:56:03,935 INFO [Listener at localhost/36281] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-08 18:56:03,937 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:03,938 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:03,941 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:03,941 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:03,960 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=248 (was 285), OpenFileDescriptor=626 (was 765), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=128 (was 139), ProcessCount=172 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=6886 (was 6900) 2023-03-08 18:56:03,974 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=128, ProcessCount=172, AvailableMemoryMB=6885 2023-03-08 18:56:03,976 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 18:56:03,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:03,977 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:56:03,978 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-08 18:56:03,978 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:56:03,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 18:56:03,979 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:03,980 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1 empty. 2023-03-08 18:56:03,980 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:03,980 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-08 18:56:04,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 18:56:04,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 18:56:04,388 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 18:56:04,389 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => fef10155ee8ca5fb6840d90511945fb1, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:56:04,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 18:56:04,794 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:56:04,794 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing fef10155ee8ca5fb6840d90511945fb1, disabling compactions & flushes 2023-03-08 18:56:04,794 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:04,794 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:04,794 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. after waiting 0 ms 2023-03-08 18:56:04,794 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:04,794 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:04,795 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:04,796 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:56:04,797 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678301764797"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301764797"}]},"ts":"1678301764797"} 2023-03-08 18:56:04,799 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 18:56:04,799 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:56:04,799 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301764799"}]},"ts":"1678301764799"} 2023-03-08 18:56:04,800 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-08 18:56:04,804 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=fef10155ee8ca5fb6840d90511945fb1, ASSIGN}] 2023-03-08 18:56:04,805 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=fef10155ee8ca5fb6840d90511945fb1, ASSIGN 2023-03-08 18:56:04,806 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=fef10155ee8ca5fb6840d90511945fb1, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:56:04,957 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=fef10155ee8ca5fb6840d90511945fb1, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:04,957 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678301764957"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301764957"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301764957"}]},"ts":"1678301764957"} 2023-03-08 18:56:04,958 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure fef10155ee8ca5fb6840d90511945fb1, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:56:05,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 18:56:05,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:05,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fef10155ee8ca5fb6840d90511945fb1, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1.', STARTKEY => '', ENDKEY => ''} 2023-03-08 18:56:05,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:05,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:56:05,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:05,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:05,114 INFO [StoreOpener-fef10155ee8ca5fb6840d90511945fb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:05,115 DEBUG [StoreOpener-fef10155ee8ca5fb6840d90511945fb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1 2023-03-08 18:56:05,115 DEBUG [StoreOpener-fef10155ee8ca5fb6840d90511945fb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1 2023-03-08 18:56:05,115 INFO [StoreOpener-fef10155ee8ca5fb6840d90511945fb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fef10155ee8ca5fb6840d90511945fb1 columnFamilyName family1 2023-03-08 18:56:05,116 INFO [StoreOpener-fef10155ee8ca5fb6840d90511945fb1-1] regionserver.HStore(310): Store=fef10155ee8ca5fb6840d90511945fb1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:56:05,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:05,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:05,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:05,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:56:05,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened fef10155ee8ca5fb6840d90511945fb1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70246047, jitterRate=0.04674766957759857}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 18:56:05,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:05,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1., pid=114, masterSystemTime=1678301765110 2023-03-08 18:56:05,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:05,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:05,123 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=fef10155ee8ca5fb6840d90511945fb1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:05,124 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678301765123"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301765123"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301765123"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301765123"}]},"ts":"1678301765123"} 2023-03-08 18:56:05,126 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-08 18:56:05,126 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure fef10155ee8ca5fb6840d90511945fb1, server=jenkins-hbase6.apache.org,46397,1678301621627 in 167 msec 2023-03-08 18:56:05,127 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-08 18:56:05,127 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=fef10155ee8ca5fb6840d90511945fb1, ASSIGN in 322 msec 2023-03-08 18:56:05,127 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:56:05,127 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301765127"}]},"ts":"1678301765127"} 2023-03-08 18:56:05,128 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-08 18:56:05,130 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:56:05,131 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1540 sec 2023-03-08 18:56:06,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 18:56:06,084 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-08 18:56:06,101 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:06,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,102 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:06,102 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:06,102 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:06,103 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:06,103 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,103 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,104 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:06,104 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:06,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:06,105 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,105 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:06,105 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:06,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,106 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,106 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:06,106 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:06,106 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:06,110 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:06,110 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:06,110 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:06,110 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:06,111 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 18:56:06,531 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130cde4359eefb34bc8ac6ce877a6b9abce to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130cde4359eefb34bc8ac6ce877a6b9abce 2023-03-08 18:56:06,934 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/c86e9de36d2d455289dd8b7bf48d0283 2023-03-08 18:56:06,938 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/c86e9de36d2d455289dd8b7bf48d0283 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c86e9de36d2d455289dd8b7bf48d0283 2023-03-08 18:56:06,942 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c86e9de36d2d455289dd8b7bf48d0283, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-08 18:56:06,942 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 831ms, sequenceid=5, compaction requested=false 2023-03-08 18:56:06,943 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:06,943 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:06,943 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:06,943 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:06,943 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:06,943 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,943 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:06,943 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:06,945 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,945 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:06,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:06,945 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,945 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:06,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:06,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:06,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@396896c0[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:06,947 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:06,947 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,949 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,949 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:06,949 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:06,949 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:06,950 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,950 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:06,951 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:06,951 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,951 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:06,951 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:06,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:06,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:06,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:06,953 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:06,953 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:06,953 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:06,953 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,953 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:06,955 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,955 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:06,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,955 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:06,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:06,955 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,956 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:06,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,958 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,958 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,958 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,958 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:06,958 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:06,958 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:06,958 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,959 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:06,959 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:06,958 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:06,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:06,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:06,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:06,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:06,959 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:06,959 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:06,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:07,059 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:07,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:07,064 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:07,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:07,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,066 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:07,066 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:07,066 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:07,066 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:07,066 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,066 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,068 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,068 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:07,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:07,068 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,068 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,069 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,069 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:07,069 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:07,069 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:07,070 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:07,070 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:07,070 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:07,070 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:07,070 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 18:56:07,480 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b7c7198ba1424faeb8d52e34675832e3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b7c7198ba1424faeb8d52e34675832e3 2023-03-08 18:56:07,886 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/408fbb0b9126467c8af8ec23030ee72f 2023-03-08 18:56:07,890 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/408fbb0b9126467c8af8ec23030ee72f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/408fbb0b9126467c8af8ec23030ee72f 2023-03-08 18:56:07,893 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/408fbb0b9126467c8af8ec23030ee72f, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 18:56:07,893 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 823ms, sequenceid=9, compaction requested=false 2023-03-08 18:56:07,893 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:07,894 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:07,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:07,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:07,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:07,894 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:07,895 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,895 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:07,896 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,896 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:07,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:07,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4aec2a7c[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:07,897 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:07,897 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,898 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,898 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:07,899 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:07,899 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:07,899 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,899 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:07,900 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:07,900 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,900 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:07,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:07,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:07,900 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:07,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:07,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:07,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:07,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:07,902 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:07,902 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:07,902 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:07,902 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,902 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:07,905 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,905 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:07,905 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:07,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:07,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:07,905 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:07,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:07,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,907 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,909 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,909 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,909 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,909 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:07,909 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:07,909 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:07,909 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:07,909 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:07,909 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:07,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:07,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:07,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:56:07,910 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:07,910 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:07,910 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:07,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:07,910 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:07,910 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,010 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:08,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:08,014 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:08,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:08,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,016 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:08,016 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:08,016 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:08,016 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:08,016 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,016 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,020 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:08,020 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:08,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:08,021 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,021 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,021 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,021 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:08,021 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,021 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:08,022 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:08,022 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:08,022 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:08,022 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:08,022 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:08,022 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:08,022 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:56:08,433 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120135f8a1bac4574db48b8ee024e769705e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120135f8a1bac4574db48b8ee024e769705e 2023-03-08 18:56:08,838 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/cc5c1cb860ef4367ad8a20285667ebf5 2023-03-08 18:56:08,842 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/cc5c1cb860ef4367ad8a20285667ebf5 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/cc5c1cb860ef4367ad8a20285667ebf5 2023-03-08 18:56:08,845 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/cc5c1cb860ef4367ad8a20285667ebf5, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 18:56:08,846 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 824ms, sequenceid=13, compaction requested=true 2023-03-08 18:56:08,846 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:08,846 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:08,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:08,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:08,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:08,846 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:08,848 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,848 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:08,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:08,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,849 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:08,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:08,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:08,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:08,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:08,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47d98ea4[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:08,850 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:08,850 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,851 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,851 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:08,851 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:08,851 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:08,851 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,851 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:08,853 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:08,853 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:08,853 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,853 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:08,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:08,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:08,855 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:08,855 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:08,855 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:08,855 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,855 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:08,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,856 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:08,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,857 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:08,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:08,857 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,858 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:08,861 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:08,861 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:08,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,861 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,861 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:08,862 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:08,962 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:08,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:08,966 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:08,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:08,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,967 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:08,967 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:08,967 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:08,967 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:08,968 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,968 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,969 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,969 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:08,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:08,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:08,969 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:08,969 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:08,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:08,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:08,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:08,970 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:08,970 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:08,970 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:08,971 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:08,971 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:08,971 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:08,971 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:08,971 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:56:09,381 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205142f4191c3b249f2b652345e147d0d5b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205142f4191c3b249f2b652345e147d0d5b 2023-03-08 18:56:09,785 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/34186200883349c5bda411d285486a66 2023-03-08 18:56:09,789 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/34186200883349c5bda411d285486a66 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/34186200883349c5bda411d285486a66 2023-03-08 18:56:09,792 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/34186200883349c5bda411d285486a66, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 18:56:09,793 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 822ms, sequenceid=17, compaction requested=true 2023-03-08 18:56:09,793 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:09,793 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:09,793 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:09,793 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:09,793 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,793 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:09,793 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:09,795 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:09,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,795 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:09,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:09,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@db9f10d[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:09,796 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:09,796 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,797 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,798 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:09,798 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:09,798 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:09,798 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,798 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:09,800 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:09,800 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,800 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:09,800 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:09,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:09,802 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:09,802 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:09,802 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:09,802 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,802 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:09,804 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,804 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:09,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,804 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:09,804 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:09,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,805 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:09,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,807 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,807 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,807 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,807 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:09,807 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:09,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:56:09,808 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:09,807 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:09,808 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:09,807 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:09,808 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:09,808 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,808 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,808 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:09,808 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:09,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:09,908 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:09,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:09,913 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:09,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:09,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,914 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:09,914 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:09,914 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:09,914 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:09,915 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,915 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,916 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,916 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:09,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:09,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:09,916 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:09,916 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:09,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:09,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,918 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:09,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:09,918 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:09,918 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:09,918 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:09,918 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:09,918 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:09,918 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:09,918 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:09,918 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:56:10,336 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512282d3fc63ccf7c4766b8dcf8a784b7b938 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512282d3fc63ccf7c4766b8dcf8a784b7b938 2023-03-08 18:56:10,648 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:56:10,741 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/b1eee33708644e51b1b5dcb94ac56fd3 2023-03-08 18:56:10,745 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/b1eee33708644e51b1b5dcb94ac56fd3 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b1eee33708644e51b1b5dcb94ac56fd3 2023-03-08 18:56:10,749 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b1eee33708644e51b1b5dcb94ac56fd3, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 18:56:10,749 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 831ms, sequenceid=21, compaction requested=true 2023-03-08 18:56:10,749 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:10,749 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:10,749 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:10,750 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:10,750 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,750 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:10,750 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:10,752 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,752 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:10,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:10,753 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,753 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:10,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:10,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:10,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:10,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:10,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@709ab8ec[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:10,754 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:10,754 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,755 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,755 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:10,755 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:10,755 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:10,756 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,756 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:10,757 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:10,757 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,757 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:10,757 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:10,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:10,759 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:10,759 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:10,759 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:10,759 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,759 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:10,760 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,760 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,760 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:10,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:10,761 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:10,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,762 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:10,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:10,768 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:10,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:10,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:10,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:10,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:10,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,768 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:10,869 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:10,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:10,872 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:10,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:10,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,874 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:10,874 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:10,874 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:10,874 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:10,875 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,875 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,876 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:10,876 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:10,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:10,876 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:10,876 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:10,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:10,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,877 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,877 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:10,877 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:10,877 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:10,877 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:10,877 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:10,877 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:10,877 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:10,877 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:10,877 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:10,878 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:56:11,288 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f8128994da424b7cb68c2877a816d22e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231f8128994da424b7cb68c2877a816d22e 2023-03-08 18:56:11,692 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/b7a9c1d7645847b8b8c1532728b0f1d3 2023-03-08 18:56:11,696 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/b7a9c1d7645847b8b8c1532728b0f1d3 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b7a9c1d7645847b8b8c1532728b0f1d3 2023-03-08 18:56:11,699 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b7a9c1d7645847b8b8c1532728b0f1d3, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 18:56:11,700 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 822ms, sequenceid=25, compaction requested=true 2023-03-08 18:56:11,700 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:11,700 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:11,700 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:11,700 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:11,700 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,700 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:11,700 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:11,702 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,702 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:11,702 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:11,702 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:11,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b0ed349[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:11,703 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:11,703 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,706 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,706 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:11,706 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:11,706 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:11,706 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,706 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:11,707 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:11,707 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:11,707 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,707 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:11,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:11,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:11,709 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:11,709 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:11,709 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:11,709 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,709 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:11,711 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,711 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:11,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,711 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:11,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:11,711 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:11,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,713 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:11,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,715 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:11,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:56:11,715 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,715 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:11,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:11,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,715 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:11,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:11,815 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:11,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:11,820 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:11,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:11,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,821 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:11,821 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:11,822 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:11,822 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:11,822 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,822 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,824 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,824 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:11,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:11,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:11,824 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:11,824 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:11,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:11,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:11,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:11,825 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:11,825 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:11,825 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:11,825 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:11,825 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:11,825 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:11,825 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:11,825 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:56:12,235 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010104bee1df75f349298ee12e503a672487 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010104bee1df75f349298ee12e503a672487 2023-03-08 18:56:12,640 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/196977c18fdf4279942d5a1f7c9e5427 2023-03-08 18:56:12,644 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/196977c18fdf4279942d5a1f7c9e5427 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/196977c18fdf4279942d5a1f7c9e5427 2023-03-08 18:56:12,647 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/196977c18fdf4279942d5a1f7c9e5427, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 18:56:12,648 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 823ms, sequenceid=29, compaction requested=true 2023-03-08 18:56:12,648 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:12,648 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:12,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:12,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:12,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:12,648 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:12,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,650 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:12,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:12,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,650 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:12,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:12,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:12,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:12,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:12,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50a9ca22[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:12,652 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:12,652 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,653 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,653 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:12,653 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:12,653 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,653 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:12,653 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:12,655 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:12,655 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:12,655 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,655 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:12,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:12,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:12,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:12,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:12,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:12,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:12,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:12,657 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:12,657 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:12,657 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:12,657 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,657 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:12,659 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,659 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:12,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:12,659 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:12,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:12,660 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:12,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:12,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:12,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,661 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:12,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:12,663 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:12,663 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:12,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:12,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:56:12,663 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,663 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:12,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:12,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:12,764 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:12,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:12,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:12,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:12,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,770 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:12,770 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:12,770 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:12,770 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:12,770 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,770 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,771 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:12,771 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:12,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:12,772 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:12,772 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:12,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,772 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:12,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,772 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,773 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:12,773 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:12,773 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:12,773 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:12,773 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:12,773 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:12,773 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:12,773 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:12,773 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:12,773 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:56:13,184 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103340b14066eb246d18f9a1e5040e103f4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103340b14066eb246d18f9a1e5040e103f4 2023-03-08 18:56:13,591 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/e11aace52a0e4a7ba6da4c53303f7198 2023-03-08 18:56:13,595 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/e11aace52a0e4a7ba6da4c53303f7198 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/e11aace52a0e4a7ba6da4c53303f7198 2023-03-08 18:56:13,597 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/e11aace52a0e4a7ba6da4c53303f7198, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 18:56:13,598 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 825ms, sequenceid=33, compaction requested=true 2023-03-08 18:56:13,598 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:13,598 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:13,598 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:13,598 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:13,598 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,598 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:13,598 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:13,600 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,600 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:13,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:13,600 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,600 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:13,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:13,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:13,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@125a78a3[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:13,601 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:13,601 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,603 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,603 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:13,603 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:13,603 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:13,604 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,604 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:13,605 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,605 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:13,605 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:13,605 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:13,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:13,607 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:13,607 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:13,607 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:13,607 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,607 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:13,608 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,609 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:13,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,609 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:13,609 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:13,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,610 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:13,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:13,616 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:13,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:13,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:13,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,616 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:13,617 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:13,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:13,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:13,717 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:13,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:13,721 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:13,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:13,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,722 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:13,722 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:13,722 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:13,722 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:13,723 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,723 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,724 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:13,724 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:13,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:13,724 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:13,724 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:13,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,725 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,725 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:13,725 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:13,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:13,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:13,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:13,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:13,725 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:13,725 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:13,725 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:13,726 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:56:13,736 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075197d5ac5e6b4324b8cec1a15cc240c5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509075197d5ac5e6b4324b8cec1a15cc240c5 2023-03-08 18:56:14,141 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/09a24be5a3264f0f9bbd657349cfadc5 2023-03-08 18:56:14,145 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/09a24be5a3264f0f9bbd657349cfadc5 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/09a24be5a3264f0f9bbd657349cfadc5 2023-03-08 18:56:14,149 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/09a24be5a3264f0f9bbd657349cfadc5, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 18:56:14,150 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 425ms, sequenceid=37, compaction requested=true 2023-03-08 18:56:14,150 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:14,150 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:14,150 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:14,150 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:14,150 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,150 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:14,150 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:14,152 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,152 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:14,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:14,152 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,152 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:14,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:14,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:14,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:14,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:14,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3722ffdc[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:14,154 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:14,154 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,156 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,156 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:14,157 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:14,156 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,157 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:14,157 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:14,158 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:14,158 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,158 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:14,158 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:14,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:14,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:14,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:14,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:14,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:14,160 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:14,160 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:14,160 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:14,160 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,160 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:14,162 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,162 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:14,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:14,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,162 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:14,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:14,163 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:14,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:14,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:14,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,164 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:14,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,166 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,166 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:14,166 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:14,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:56:14,166 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:14,166 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:14,166 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:14,166 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:14,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:14,166 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,166 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,167 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,167 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,167 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,167 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:14,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:14,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:14,167 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:14,267 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:14,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:14,271 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:14,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:14,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,272 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:14,273 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:14,273 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 18:56:14,273 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 18:56:14,273 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,273 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,279 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:14,279 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:14,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:14,279 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:14,279 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:14,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:14,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 18:56:14,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:14,280 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 18:56:14,280 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:14,280 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 18:56:14,280 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 18:56:14,280 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:14,280 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:14,281 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. started... 2023-03-08 18:56:14,281 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fef10155ee8ca5fb6840d90511945fb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 18:56:14,693 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511209dd5f46b31794c41a3abfb883d6e3875 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511209dd5f46b31794c41a3abfb883d6e3875 2023-03-08 18:56:15,098 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/731c3c31763f43318c9d564064dc3da6 2023-03-08 18:56:15,102 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/731c3c31763f43318c9d564064dc3da6 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/731c3c31763f43318c9d564064dc3da6 2023-03-08 18:56:15,105 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/731c3c31763f43318c9d564064dc3da6, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 18:56:15,105 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fef10155ee8ca5fb6840d90511945fb1 in 824ms, sequenceid=41, compaction requested=true 2023-03-08 18:56:15,105 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:15,106 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:15,106 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 18:56:15,106 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 18:56:15,106 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,106 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 18:56:15,106 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 18:56:15,108 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,108 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:15,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:15,108 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:15,108 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:15,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:15,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:15,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 18:56:15,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a1bf6ea[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:15,109 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 18:56:15,109 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,111 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,111 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 18:56:15,111 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 18:56:15,111 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:15,111 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,111 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:15,112 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 18:56:15,112 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,112 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:15,112 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 18:56:15,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 18:56:15,114 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 18:56:15,114 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:15,114 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:15,114 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,114 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:15,117 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,117 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:15,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:15,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,117 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:15,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:15,117 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:15,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:15,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:15,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,118 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:15,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,120 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:15,120 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:15,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 18:56:15,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:15,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:15,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 18:56:15,120 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:15,120 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,120 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:15,121 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,121 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:15,121 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:15,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:15,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:15,121 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,121 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:15,121 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,121 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,221 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:15,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:15,223 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-08 18:56:15,224 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:56:15,225 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:56:15,226 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:56:15,226 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302375226 2023-03-08 18:56:15,226 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 18:56:15,227 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-08 18:56:15,227 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:56:15,227 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x61d18c9e to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:56:15,231 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1ca61305, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:56:15,232 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:56:16,062 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031ed3eecdc20a483c82debb1d7a7d8c40 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601031ed3eecdc20a483c82debb1d7a7d8c40 2023-03-08 18:56:16,063 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:56:16,075 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:56:16,077 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:50258, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:56:16,082 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:56:16,083 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:44698, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:56:16,090 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/a0d7373ec92a4a9fb6c527268795411f first=Optional[r5] last=Optional[r6] 2023-03-08 18:56:16,091 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/a0d7373ec92a4a9fb6c527268795411f}] 2023-03-08 18:56:16,097 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/a0d7373ec92a4a9fb6c527268795411f for inclusion in fef10155ee8ca5fb6840d90511945fb1/family1 2023-03-08 18:56:16,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-08 18:56:16,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:56:16,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:16,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/a0d7373ec92a4a9fb6c527268795411f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__82qqoebvm2sgs60ba1nte5o49urlq7b45ucau0noea8t3ennsivoili64kkgc0uu/family1/a0d7373ec92a4a9fb6c527268795411f 2023-03-08 18:56:16,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__82qqoebvm2sgs60ba1nte5o49urlq7b45ucau0noea8t3ennsivoili64kkgc0uu/family1/a0d7373ec92a4a9fb6c527268795411f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/6167810255e04314b6e7c0accb8537ed_SeqId_44_ 2023-03-08 18:56:16,103 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__82qqoebvm2sgs60ba1nte5o49urlq7b45ucau0noea8t3ennsivoili64kkgc0uu/family1/a0d7373ec92a4a9fb6c527268795411f into fef10155ee8ca5fb6840d90511945fb1/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/6167810255e04314b6e7c0accb8537ed_SeqId_44_ - updating store file list. 2023-03-08 18:56:16,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/6167810255e04314b6e7c0accb8537ed_SeqId_44_ into fef10155ee8ca5fb6840d90511945fb1/family1 2023-03-08 18:56:16,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__82qqoebvm2sgs60ba1nte5o49urlq7b45ucau0noea8t3ennsivoili64kkgc0uu/family1/a0d7373ec92a4a9fb6c527268795411f into fef10155ee8ca5fb6840d90511945fb1/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/6167810255e04314b6e7c0accb8537ed_SeqId_44_) 2023-03-08 18:56:16,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__82qqoebvm2sgs60ba1nte5o49urlq7b45ucau0noea8t3ennsivoili64kkgc0uu/family1/a0d7373ec92a4a9fb6c527268795411f 2023-03-08 18:56:16,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 18:56:16,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 18:56:16,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(1912): fef10155ee8ca5fb6840d90511945fb1/family1 is initiating minor compaction 2023-03-08 18:56:16,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@56921da6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:56:16,110 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fef10155ee8ca5fb6840d90511945fb1/family1 in testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:16,110 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/408fbb0b9126467c8af8ec23030ee72f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/cc5c1cb860ef4367ad8a20285667ebf5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/34186200883349c5bda411d285486a66, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b1eee33708644e51b1b5dcb94ac56fd3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b7a9c1d7645847b8b8c1532728b0f1d3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/196977c18fdf4279942d5a1f7c9e5427, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/e11aace52a0e4a7ba6da4c53303f7198, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/09a24be5a3264f0f9bbd657349cfadc5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/731c3c31763f43318c9d564064dc3da6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/6167810255e04314b6e7c0accb8537ed_SeqId_44_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp, totalSize=49.7 K 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 408fbb0b9126467c8af8ec23030ee72f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting cc5c1cb860ef4367ad8a20285667ebf5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 34186200883349c5bda411d285486a66, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting b1eee33708644e51b1b5dcb94ac56fd3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting b7a9c1d7645847b8b8c1532728b0f1d3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 196977c18fdf4279942d5a1f7c9e5427, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting e11aace52a0e4a7ba6da4c53303f7198, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 09a24be5a3264f0f9bbd657349cfadc5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 731c3c31763f43318c9d564064dc3da6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 18:56:16,110 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 6167810255e04314b6e7c0accb8537ed_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 18:56:16,111 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:56:16,112 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:16,113 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010104bee1df75f349298ee12e503a672487 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010104bee1df75f349298ee12e503a672487 2023-03-08 18:56:16,114 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103340b14066eb246d18f9a1e5040e103f4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103340b14066eb246d18f9a1e5040e103f4 2023-03-08 18:56:16,114 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-08 18:56:16,114 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-08 18:56:16,115 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:56:16,115 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:56:16,146 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): fef10155ee8ca5fb6840d90511945fb1#family1#compaction#166 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:16,964 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/c884d339acc64a4dbc3048466a892f41 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c884d339acc64a4dbc3048466a892f41 2023-03-08 18:56:16,966 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231b131a4b807c740cd942ee852ceaf3d41 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231b131a4b807c740cd942ee852ceaf3d41 2023-03-08 18:56:16,967 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:56:16,969 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in fef10155ee8ca5fb6840d90511945fb1/family1 of fef10155ee8ca5fb6840d90511945fb1 into c884d339acc64a4dbc3048466a892f41(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:56:16,969 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:16,969 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678301776109; duration=0sec 2023-03-08 18:56:16,969 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:56:16,982 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/408fbb0b9126467c8af8ec23030ee72f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/cc5c1cb860ef4367ad8a20285667ebf5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/34186200883349c5bda411d285486a66, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b1eee33708644e51b1b5dcb94ac56fd3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b7a9c1d7645847b8b8c1532728b0f1d3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/196977c18fdf4279942d5a1f7c9e5427, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/e11aace52a0e4a7ba6da4c53303f7198, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/09a24be5a3264f0f9bbd657349cfadc5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/731c3c31763f43318c9d564064dc3da6, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/6167810255e04314b6e7c0accb8537ed_SeqId_44_] to archive 2023-03-08 18:56:16,982 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:16,985 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/408fbb0b9126467c8af8ec23030ee72f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/408fbb0b9126467c8af8ec23030ee72f 2023-03-08 18:56:16,987 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/cc5c1cb860ef4367ad8a20285667ebf5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/cc5c1cb860ef4367ad8a20285667ebf5 2023-03-08 18:56:16,988 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/34186200883349c5bda411d285486a66 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/34186200883349c5bda411d285486a66 2023-03-08 18:56:16,990 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b1eee33708644e51b1b5dcb94ac56fd3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b1eee33708644e51b1b5dcb94ac56fd3 2023-03-08 18:56:16,992 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b7a9c1d7645847b8b8c1532728b0f1d3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/b7a9c1d7645847b8b8c1532728b0f1d3 2023-03-08 18:56:16,993 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/93b67c11408d433c8a7a76713025fb17 first=Optional[r1] last=Optional[r4] 2023-03-08 18:56:16,993 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/93b67c11408d433c8a7a76713025fb17}] 2023-03-08 18:56:16,994 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/196977c18fdf4279942d5a1f7c9e5427 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/196977c18fdf4279942d5a1f7c9e5427 2023-03-08 18:56:16,995 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/e11aace52a0e4a7ba6da4c53303f7198 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/e11aace52a0e4a7ba6da4c53303f7198 2023-03-08 18:56:16,995 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/93b67c11408d433c8a7a76713025fb17 for inclusion in fef10155ee8ca5fb6840d90511945fb1/family1 2023-03-08 18:56:16,996 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/09a24be5a3264f0f9bbd657349cfadc5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/09a24be5a3264f0f9bbd657349cfadc5 2023-03-08 18:56:16,997 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/731c3c31763f43318c9d564064dc3da6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/731c3c31763f43318c9d564064dc3da6 2023-03-08 18:56:16,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-08 18:56:16,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:56:16,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:16,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/93b67c11408d433c8a7a76713025fb17 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2i9dtqf8eo8opvro0jpet4atle893h4s7lvq8uf6e01vlrc6fkrang431ji4odui/family1/93b67c11408d433c8a7a76713025fb17 2023-03-08 18:56:16,999 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/6167810255e04314b6e7c0accb8537ed_SeqId_44_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/6167810255e04314b6e7c0accb8537ed_SeqId_44_ 2023-03-08 18:56:17,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2i9dtqf8eo8opvro0jpet4atle893h4s7lvq8uf6e01vlrc6fkrang431ji4odui/family1/93b67c11408d433c8a7a76713025fb17 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/4fe7a9d80061430e8831599154e302b7_SeqId_47_ 2023-03-08 18:56:17,001 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2i9dtqf8eo8opvro0jpet4atle893h4s7lvq8uf6e01vlrc6fkrang431ji4odui/family1/93b67c11408d433c8a7a76713025fb17 into fef10155ee8ca5fb6840d90511945fb1/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/4fe7a9d80061430e8831599154e302b7_SeqId_47_ - updating store file list. 2023-03-08 18:56:17,005 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/4fe7a9d80061430e8831599154e302b7_SeqId_47_ into fef10155ee8ca5fb6840d90511945fb1/family1 2023-03-08 18:56:17,005 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2i9dtqf8eo8opvro0jpet4atle893h4s7lvq8uf6e01vlrc6fkrang431ji4odui/family1/93b67c11408d433c8a7a76713025fb17 into fef10155ee8ca5fb6840d90511945fb1/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/4fe7a9d80061430e8831599154e302b7_SeqId_47_) 2023-03-08 18:56:17,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__2i9dtqf8eo8opvro0jpet4atle893h4s7lvq8uf6e01vlrc6fkrang431ji4odui/family1/93b67c11408d433c8a7a76713025fb17 2023-03-08 18:56:17,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 18:56:17,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 18:56:17,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(1912): fef10155ee8ca5fb6840d90511945fb1/family1 is initiating minor compaction (all files) 2023-03-08 18:56:17,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4eb60fbe; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:56:17,006 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fef10155ee8ca5fb6840d90511945fb1/family1 in testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:17,006 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c86e9de36d2d455289dd8b7bf48d0283, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c884d339acc64a4dbc3048466a892f41, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/4fe7a9d80061430e8831599154e302b7_SeqId_47_] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp, totalSize=178.1 K 2023-03-08 18:56:17,007 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting c86e9de36d2d455289dd8b7bf48d0283, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845940 2023-03-08 18:56:17,007 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting c884d339acc64a4dbc3048466a892f41, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245940 2023-03-08 18:56:17,007 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] compactions.Compactor(207): Compacting 4fe7a9d80061430e8831599154e302b7_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965245940 2023-03-08 18:56:17,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:56:17,008 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:17,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120135f8a1bac4574db48b8ee024e769705e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120135f8a1bac4574db48b8ee024e769705e 2023-03-08 18:56:17,011 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205142f4191c3b249f2b652345e147d0d5b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205142f4191c3b249f2b652345e147d0d5b 2023-03-08 18:56:17,012 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512282d3fc63ccf7c4766b8dcf8a784b7b938 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512282d3fc63ccf7c4766b8dcf8a784b7b938 2023-03-08 18:56:17,013 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231f8128994da424b7cb68c2877a816d22e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231f8128994da424b7cb68c2877a816d22e 2023-03-08 18:56:17,013 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-08 18:56:17,013 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-08 18:56:17,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:56:17,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:56:17,019 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] throttle.PressureAwareThroughputController(145): fef10155ee8ca5fb6840d90511945fb1#family1#compaction#167 average throughput is 31.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:17,460 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130ff3a7064b4094dfc97c6fac6df78a78d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130ff3a7064b4094dfc97c6fac6df78a78d 2023-03-08 18:56:17,460 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:56:17,482 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7c4f6c15a14d4f4d896df9f0a640189d first=Optional[r00] last=Optional[r8] 2023-03-08 18:56:17,482 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7c4f6c15a14d4f4d896df9f0a640189d}] 2023-03-08 18:56:17,484 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7c4f6c15a14d4f4d896df9f0a640189d for inclusion in fef10155ee8ca5fb6840d90511945fb1/family1 2023-03-08 18:56:17,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-08 18:56:17,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 18:56:17,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:17,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7c4f6c15a14d4f4d896df9f0a640189d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6udk8lvnrenuga80ldku8thih3esej0rk6d25f4or8k36vgkhaqs9ebg4ph4cmoe/family1/7c4f6c15a14d4f4d896df9f0a640189d 2023-03-08 18:56:17,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6udk8lvnrenuga80ldku8thih3esej0rk6d25f4or8k36vgkhaqs9ebg4ph4cmoe/family1/7c4f6c15a14d4f4d896df9f0a640189d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/bff438bc2b0d49c8badf0399a28d0190_SeqId_49_ 2023-03-08 18:56:17,489 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6udk8lvnrenuga80ldku8thih3esej0rk6d25f4or8k36vgkhaqs9ebg4ph4cmoe/family1/7c4f6c15a14d4f4d896df9f0a640189d into fef10155ee8ca5fb6840d90511945fb1/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/bff438bc2b0d49c8badf0399a28d0190_SeqId_49_ - updating store file list. 2023-03-08 18:56:17,491 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/bff438bc2b0d49c8badf0399a28d0190_SeqId_49_ into fef10155ee8ca5fb6840d90511945fb1/family1 2023-03-08 18:56:17,491 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6udk8lvnrenuga80ldku8thih3esej0rk6d25f4or8k36vgkhaqs9ebg4ph4cmoe/family1/7c4f6c15a14d4f4d896df9f0a640189d into fef10155ee8ca5fb6840d90511945fb1/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/bff438bc2b0d49c8badf0399a28d0190_SeqId_49_) 2023-03-08 18:56:17,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6udk8lvnrenuga80ldku8thih3esej0rk6d25f4or8k36vgkhaqs9ebg4ph4cmoe/family1/7c4f6c15a14d4f4d896df9f0a640189d 2023-03-08 18:56:17,493 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:56:17,493 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:17,495 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511209dd5f46b31794c41a3abfb883d6e3875 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511209dd5f46b31794c41a3abfb883d6e3875 2023-03-08 18:56:17,495 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b7c7198ba1424faeb8d52e34675832e3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b7c7198ba1424faeb8d52e34675832e3 2023-03-08 18:56:17,496 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130cde4359eefb34bc8ac6ce877a6b9abce to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130cde4359eefb34bc8ac6ce877a6b9abce 2023-03-08 18:56:17,496 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-08 18:56:17,496 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 18:56:17,496 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:56:17,496 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:56:17,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 18:56:17,497 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 18:56:17,497 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:56:17,499 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 18:56:17,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.2750 sec 2023-03-08 18:56:17,711 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=257 (was 248) - Thread LEAK? -, OpenFileDescriptor=645 (was 626) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 128) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 172), AvailableMemoryMB=6878 (was 6885) 2023-03-08 18:56:17,726 INFO [Listener at localhost/36281] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=257, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=196, ProcessCount=170, AvailableMemoryMB=6877 2023-03-08 18:56:17,746 DEBUG [Listener at localhost/36281] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-08 18:56:17,814 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.HMaster$4(2112): Client=jenkins//172.31.2.20 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10#\x9B\x1C\x12\x95T\xC8\xF7\xC5\x91\x84\x08\x18\xF0\xE2\xD4"\x10\xA5\x19\xA8(\xDC\xEB\x94~O\xBD\xFAz\xEF\xE6\xD2\x9E*\x10=\x9DQ \x81\x1AR\xE8\xCD\xCEff\x8C\x5CWG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 18:56:17,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-08 18:56:17,817 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 18:56:17,817 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(697): Client=jenkins//172.31.2.20 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-08 18:56:17,818 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 18:56:17,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 18:56:17,820 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:17,820 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:17,820 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:17,820 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a empty. 2023-03-08 18:56:17,820 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e empty. 2023-03-08 18:56:17,820 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f empty. 2023-03-08 18:56:17,820 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:17,821 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:17,821 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:17,821 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-08 18:56:17,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 18:56:18,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 18:56:18,229 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-08 18:56:18,231 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => f3e546ab6e95b3c35d94bf0350799d5a, NAME => 'testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10#\x9B\x1C\x12\x95T\xC8\xF7\xC5\x91\x84\x08\x18\xF0\xE2\xD4"\x10\xA5\x19\xA8(\xDC\xEB\x94~O\xBD\xFAz\xEF\xE6\xD2\x9E*\x10=\x9DQ \x81\x1AR\xE8\xCD\xCEff\x8C\x5CWG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:56:18,231 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 7c4e5be8c7bf7634fdec5582cd8f9f5e, NAME => 'testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10#\x9B\x1C\x12\x95T\xC8\xF7\xC5\x91\x84\x08\x18\xF0\xE2\xD4"\x10\xA5\x19\xA8(\xDC\xEB\x94~O\xBD\xFAz\xEF\xE6\xD2\x9E*\x10=\x9DQ \x81\x1AR\xE8\xCD\xCEff\x8C\x5CWG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:56:18,231 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 8b7ffc3b5de37aaac329ae67ecaf3e9f, NAME => 'testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10#\x9B\x1C\x12\x95T\xC8\xF7\xC5\x91\x84\x08\x18\xF0\xE2\xD4"\x10\xA5\x19\xA8(\xDC\xEB\x94~O\xBD\xFAz\xEF\xE6\xD2\x9E*\x10=\x9DQ \x81\x1AR\xE8\xCD\xCEff\x8C\x5CWG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/.tmp 2023-03-08 18:56:18,240 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:56:18,240 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing f3e546ab6e95b3c35d94bf0350799d5a, disabling compactions & flushes 2023-03-08 18:56:18,241 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:18,241 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:18,241 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. after waiting 0 ms 2023-03-08 18:56:18,241 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:18,241 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:18,241 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:18,251 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/.tmp/family1/08048c8e16ba4c419bab6639846d7249 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/08048c8e16ba4c419bab6639846d7249 2023-03-08 18:56:18,255 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in fef10155ee8ca5fb6840d90511945fb1/family1 of fef10155ee8ca5fb6840d90511945fb1 into 08048c8e16ba4c419bab6639846d7249(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 18:56:18,255 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:18,255 INFO [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678301777006; duration=1sec 2023-03-08 18:56:18,255 DEBUG [RS:0;jenkins-hbase6:46397-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 18:56:18,281 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c86e9de36d2d455289dd8b7bf48d0283, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c884d339acc64a4dbc3048466a892f41, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/4fe7a9d80061430e8831599154e302b7_SeqId_47_] to archive 2023-03-08 18:56:18,282 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:18,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c86e9de36d2d455289dd8b7bf48d0283 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c86e9de36d2d455289dd8b7bf48d0283 2023-03-08 18:56:18,284 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c884d339acc64a4dbc3048466a892f41 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/c884d339acc64a4dbc3048466a892f41 2023-03-08 18:56:18,285 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/4fe7a9d80061430e8831599154e302b7_SeqId_47_ to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/family1/4fe7a9d80061430e8831599154e302b7_SeqId_47_ 2023-03-08 18:56:18,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 18:56:18,637 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:56:18,638 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 8b7ffc3b5de37aaac329ae67ecaf3e9f, disabling compactions & flushes 2023-03-08 18:56:18,638 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:18,638 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:18,638 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. after waiting 0 ms 2023-03-08 18:56:18,638 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:18,638 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:18,638 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:18,641 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:56:18,641 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 7c4e5be8c7bf7634fdec5582cd8f9f5e, disabling compactions & flushes 2023-03-08 18:56:18,641 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:18,641 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:18,641 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. after waiting 0 ms 2023-03-08 18:56:18,641 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:18,641 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:18,641 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:18,643 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 18:56:18,644 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301778644"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301778644"}]},"ts":"1678301778644"} 2023-03-08 18:56:18,644 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301778644"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301778644"}]},"ts":"1678301778644"} 2023-03-08 18:56:18,644 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678301778644"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678301778644"}]},"ts":"1678301778644"} 2023-03-08 18:56:18,646 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 18:56:18,646 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 18:56:18,646 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301778646"}]},"ts":"1678301778646"} 2023-03-08 18:56:18,647 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-08 18:56:18,650 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f3e546ab6e95b3c35d94bf0350799d5a, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=7c4e5be8c7bf7634fdec5582cd8f9f5e, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8b7ffc3b5de37aaac329ae67ecaf3e9f, ASSIGN}] 2023-03-08 18:56:18,652 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8b7ffc3b5de37aaac329ae67ecaf3e9f, ASSIGN 2023-03-08 18:56:18,652 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=7c4e5be8c7bf7634fdec5582cd8f9f5e, ASSIGN 2023-03-08 18:56:18,652 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f3e546ab6e95b3c35d94bf0350799d5a, ASSIGN 2023-03-08 18:56:18,652 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8b7ffc3b5de37aaac329ae67ecaf3e9f, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:56:18,652 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f3e546ab6e95b3c35d94bf0350799d5a, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:56:18,652 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=7c4e5be8c7bf7634fdec5582cd8f9f5e, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,46397,1678301621627; forceNewPlan=false, retain=false 2023-03-08 18:56:18,804 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=7c4e5be8c7bf7634fdec5582cd8f9f5e, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:18,804 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=8b7ffc3b5de37aaac329ae67ecaf3e9f, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:18,804 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678301778804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301778804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301778804"}]},"ts":"1678301778804"} 2023-03-08 18:56:18,804 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=f3e546ab6e95b3c35d94bf0350799d5a, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:18,804 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301778804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301778804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301778804"}]},"ts":"1678301778804"} 2023-03-08 18:56:18,804 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301778804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678301778804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678301778804"}]},"ts":"1678301778804"} 2023-03-08 18:56:18,806 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=118, state=RUNNABLE; OpenRegionProcedure 7c4e5be8c7bf7634fdec5582cd8f9f5e, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:56:18,806 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=119, state=RUNNABLE; OpenRegionProcedure 8b7ffc3b5de37aaac329ae67ecaf3e9f, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:56:18,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=117, state=RUNNABLE; OpenRegionProcedure f3e546ab6e95b3c35d94bf0350799d5a, server=jenkins-hbase6.apache.org,46397,1678301621627}] 2023-03-08 18:56:18,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 18:56:18,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:18,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8b7ffc3b5de37aaac329ae67ecaf3e9f, NAME => 'testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 18:56:18,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:18,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:56:18,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:18,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:18,982 INFO [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:18,983 DEBUG [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 2023-03-08 18:56:18,983 DEBUG [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 2023-03-08 18:56:18,983 INFO [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8b7ffc3b5de37aaac329ae67ecaf3e9f columnFamilyName family1 2023-03-08 18:56:18,984 INFO [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] regionserver.HStore(310): Store=8b7ffc3b5de37aaac329ae67ecaf3e9f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:56:18,984 INFO [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:18,985 DEBUG [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2 2023-03-08 18:56:18,985 DEBUG [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2 2023-03-08 18:56:18,986 INFO [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8b7ffc3b5de37aaac329ae67ecaf3e9f columnFamilyName family2 2023-03-08 18:56:18,987 INFO [StoreOpener-8b7ffc3b5de37aaac329ae67ecaf3e9f-1] regionserver.HStore(310): Store=8b7ffc3b5de37aaac329ae67ecaf3e9f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:56:18,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:18,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:18,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:56:18,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:18,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:56:18,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 8b7ffc3b5de37aaac329ae67ecaf3e9f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62472524, jitterRate=-0.06908684968948364}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:56:18,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:18,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f., pid=121, masterSystemTime=1678301778957 2023-03-08 18:56:18,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:18,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:18,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:18,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7c4e5be8c7bf7634fdec5582cd8f9f5e, NAME => 'testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 18:56:18,993 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=8b7ffc3b5de37aaac329ae67ecaf3e9f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:18,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:18,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:56:18,993 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301778993"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301778993"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301778993"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301778993"}]},"ts":"1678301778993"} 2023-03-08 18:56:18,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:18,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:18,996 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-03-08 18:56:18,996 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; OpenRegionProcedure 8b7ffc3b5de37aaac329ae67ecaf3e9f, server=jenkins-hbase6.apache.org,46397,1678301621627 in 188 msec 2023-03-08 18:56:18,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8b7ffc3b5de37aaac329ae67ecaf3e9f, ASSIGN in 346 msec 2023-03-08 18:56:19,015 INFO [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:19,016 DEBUG [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 2023-03-08 18:56:19,016 DEBUG [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 2023-03-08 18:56:19,017 INFO [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c4e5be8c7bf7634fdec5582cd8f9f5e columnFamilyName family1 2023-03-08 18:56:19,017 INFO [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] regionserver.HStore(310): Store=7c4e5be8c7bf7634fdec5582cd8f9f5e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:56:19,018 INFO [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:19,019 DEBUG [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2 2023-03-08 18:56:19,019 DEBUG [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2 2023-03-08 18:56:19,019 INFO [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c4e5be8c7bf7634fdec5582cd8f9f5e columnFamilyName family2 2023-03-08 18:56:19,019 INFO [StoreOpener-7c4e5be8c7bf7634fdec5582cd8f9f5e-1] regionserver.HStore(310): Store=7c4e5be8c7bf7634fdec5582cd8f9f5e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:56:19,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:19,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:19,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:56:19,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:19,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:56:19,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 7c4e5be8c7bf7634fdec5582cd8f9f5e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60942173, jitterRate=-0.0918908566236496}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:56:19,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:19,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e., pid=120, masterSystemTime=1678301778957 2023-03-08 18:56:19,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:19,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:19,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f3e546ab6e95b3c35d94bf0350799d5a, NAME => 'testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 18:56:19,026 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=7c4e5be8c7bf7634fdec5582cd8f9f5e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:19,026 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678301779026"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301779026"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301779026"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301779026"}]},"ts":"1678301779026"} 2023-03-08 18:56:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 18:56:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:19,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:19,028 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=118 2023-03-08 18:56:19,028 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=118, state=SUCCESS; OpenRegionProcedure 7c4e5be8c7bf7634fdec5582cd8f9f5e, server=jenkins-hbase6.apache.org,46397,1678301621627 in 221 msec 2023-03-08 18:56:19,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=7c4e5be8c7bf7634fdec5582cd8f9f5e, ASSIGN in 378 msec 2023-03-08 18:56:19,047 INFO [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:19,048 DEBUG [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1 2023-03-08 18:56:19,048 DEBUG [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1 2023-03-08 18:56:19,049 INFO [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3e546ab6e95b3c35d94bf0350799d5a columnFamilyName family1 2023-03-08 18:56:19,049 INFO [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] regionserver.HStore(310): Store=f3e546ab6e95b3c35d94bf0350799d5a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:56:19,049 INFO [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:19,050 DEBUG [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2 2023-03-08 18:56:19,050 DEBUG [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2 2023-03-08 18:56:19,051 INFO [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3e546ab6e95b3c35d94bf0350799d5a columnFamilyName family2 2023-03-08 18:56:19,051 INFO [StoreOpener-f3e546ab6e95b3c35d94bf0350799d5a-1] regionserver.HStore(310): Store=f3e546ab6e95b3c35d94bf0350799d5a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 18:56:19,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:19,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:19,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 18:56:19,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:19,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 18:56:19,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened f3e546ab6e95b3c35d94bf0350799d5a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61995260, jitterRate=-0.07619863748550415}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 18:56:19,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:19,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a., pid=122, masterSystemTime=1678301778957 2023-03-08 18:56:19,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:19,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:19,058 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=f3e546ab6e95b3c35d94bf0350799d5a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:19,059 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678301779058"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678301779058"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678301779058"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678301779058"}]},"ts":"1678301779058"} 2023-03-08 18:56:19,061 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=117 2023-03-08 18:56:19,061 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=117, state=SUCCESS; OpenRegionProcedure f3e546ab6e95b3c35d94bf0350799d5a, server=jenkins-hbase6.apache.org,46397,1678301621627 in 253 msec 2023-03-08 18:56:19,062 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-03-08 18:56:19,062 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f3e546ab6e95b3c35d94bf0350799d5a, ASSIGN in 411 msec 2023-03-08 18:56:19,062 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 18:56:19,062 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678301779062"}]},"ts":"1678301779062"} 2023-03-08 18:56:19,063 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-08 18:56:19,065 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 18:56:19,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2510 sec 2023-03-08 18:56:19,626 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-08 18:56:19,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 18:56:19,923 INFO [Listener at localhost/36281] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-08 18:56:19,930 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:19,930 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:19,931 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:19,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:19,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 18:56:19,934 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 18:56:19,934 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:19,934 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 18:56:19,934 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 18:56:19,935 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:19,935 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:19,937 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:19,937 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:19,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:19,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:19,938 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:19,938 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:19,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:19,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:19,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:19,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:19,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 18:56:19,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 18:56:19,939 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 18:56:19,939 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:19,939 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 18:56:19,939 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:19,939 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. started... 2023-03-08 18:56:19,939 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:19,939 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:56:19,940 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:19,940 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7c4e5be8c7bf7634fdec5582cd8f9f5e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:56:19,940 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. started... 2023-03-08 18:56:19,940 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:19,939 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. started... 2023-03-08 18:56:19,940 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8b7ffc3b5de37aaac329ae67ecaf3e9f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:56:19,941 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f3e546ab6e95b3c35d94bf0350799d5a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 18:56:20,391 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030878c6c7c73b0544499d10ba392a7eeb06 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030878c6c7c73b0544499d10ba392a7eeb06 2023-03-08 18:56:20,397 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308b419575fda624d3888a087fbe827473e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308b419575fda624d3888a087fbe827473e 2023-03-08 18:56:20,397 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303088d5b9d829f704e52b6a0c3afd234ac5e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088d5b9d829f704e52b6a0c3afd234ac5e 2023-03-08 18:56:20,810 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/cac306320bab4be9b764764628b3a028 2023-03-08 18:56:20,812 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/2480abcb54af474ba2e0c5955c42f05a 2023-03-08 18:56:20,814 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/0a76e77c02b346c6977cdefc27112a9b 2023-03-08 18:56:21,244 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303080d5792410128409ca32467d20c0ba674 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303080d5792410128409ca32467d20c0ba674 2023-03-08 18:56:21,245 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308f669bd502b6e4fc7b64bc034b3e691bd to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308f669bd502b6e4fc7b64bc034b3e691bd 2023-03-08 18:56:21,249 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/f267406f52a24517a856e7dda045db47 2023-03-08 18:56:21,250 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/faa2f88a3b4a4a6dbc8a9885ca894a82 2023-03-08 18:56:21,251 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308e82df587bbcd4c7ebae413e95a7f2236 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308e82df587bbcd4c7ebae413e95a7f2236 2023-03-08 18:56:21,254 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/cac306320bab4be9b764764628b3a028 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/cac306320bab4be9b764764628b3a028 2023-03-08 18:56:21,255 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/2480abcb54af474ba2e0c5955c42f05a as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/2480abcb54af474ba2e0c5955c42f05a 2023-03-08 18:56:21,257 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/d941504510614fdfa609225803ca5ce2 2023-03-08 18:56:21,260 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/0a76e77c02b346c6977cdefc27112a9b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/0a76e77c02b346c6977cdefc27112a9b 2023-03-08 18:56:21,284 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/cac306320bab4be9b764764628b3a028, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 18:56:21,285 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/f267406f52a24517a856e7dda045db47 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/f267406f52a24517a856e7dda045db47 2023-03-08 18:56:21,288 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/0a76e77c02b346c6977cdefc27112a9b, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 18:56:21,289 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/d941504510614fdfa609225803ca5ce2 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/d941504510614fdfa609225803ca5ce2 2023-03-08 18:56:21,290 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/2480abcb54af474ba2e0c5955c42f05a, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 18:56:21,290 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/f267406f52a24517a856e7dda045db47, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:56:21,291 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/faa2f88a3b4a4a6dbc8a9885ca894a82 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/faa2f88a3b4a4a6dbc8a9885ca894a82 2023-03-08 18:56:21,291 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f3e546ab6e95b3c35d94bf0350799d5a in 1350ms, sequenceid=5, compaction requested=false 2023-03-08 18:56:21,292 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:21,292 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:21,292 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:56:21,295 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/d941504510614fdfa609225803ca5ce2, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:56:21,295 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/faa2f88a3b4a4a6dbc8a9885ca894a82, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 18:56:21,296 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 8b7ffc3b5de37aaac329ae67ecaf3e9f in 1355ms, sequenceid=5, compaction requested=false 2023-03-08 18:56:21,296 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:21,296 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:21,296 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:56:21,296 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 7c4e5be8c7bf7634fdec5582cd8f9f5e in 1356ms, sequenceid=5, compaction requested=false 2023-03-08 18:56:21,296 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:21,296 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:21,296 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:56:21,296 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:56:21,296 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,296 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 18:56:21,296 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 18:56:21,303 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,303 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:21,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:21,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:21,304 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:21,304 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:21,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:21,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:21,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:21,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:21,305 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 18:56:21,305 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47dc751[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:21,305 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 18:56:21,305 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:21,307 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:21,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:21,307 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 18:56:21,307 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 18:56:21,307 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:21,307 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,307 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:21,308 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,308 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 18:56:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:21,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:21,308 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:21,309 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 18:56:21,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:21,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:21,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:21,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:21,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:21,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:21,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 18:56:21,311 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 18:56:21,311 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:21,311 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:21,311 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 18:56:21,311 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:21,313 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,313 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:21,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:21,313 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:21,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,313 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:21,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:21,315 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:21,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:21,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:21,318 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,318 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:21,318 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:21,318 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:21,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:21,318 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,318 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 18:56:21,319 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:21,318 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:21,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:21,318 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:21,318 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:21,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 18:56:21,319 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:21,319 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:21,319 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:21,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:21,319 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:21,419 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:21,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:21,421 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:21,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:21,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:21,424 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:21,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:21,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 18:56:21,426 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 18:56:21,426 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:21,427 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 18:56:21,427 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 18:56:21,427 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,427 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:21,428 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:21,428 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:21,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:21,429 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:21,429 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:21,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:21,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:21,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:21,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:21,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 18:56:21,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 18:56:21,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 18:56:21,430 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 18:56:21,430 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:21,430 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 18:56:21,430 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:21,430 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. started... 2023-03-08 18:56:21,430 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:21,430 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:21,430 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7c4e5be8c7bf7634fdec5582cd8f9f5e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:56:21,430 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:56:21,430 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:21,430 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. started... 2023-03-08 18:56:21,430 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. started... 2023-03-08 18:56:21,430 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8b7ffc3b5de37aaac329ae67ecaf3e9f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:56:21,430 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f3e546ab6e95b3c35d94bf0350799d5a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 18:56:21,875 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303087a84c57fb6d24aed881cbe27076f625f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303087a84c57fb6d24aed881cbe27076f625f 2023-03-08 18:56:21,876 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c3d95e05e42540968bb8295e34877224 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308c3d95e05e42540968bb8295e34877224 2023-03-08 18:56:21,876 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303082a7935cb8d2d4d20a0ffc063894dba71 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303082a7935cb8d2d4d20a0ffc063894dba71 2023-03-08 18:56:21,890 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/b7eb2b55c5ee4ffa80349f34634cf1fe 2023-03-08 18:56:22,296 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/844a6729542349ca9abf79fc516e25d4 2023-03-08 18:56:22,296 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/4c2b4c0863f5498e86047411f5545b4f 2023-03-08 18:56:22,332 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303088689dfe895bc41b1895f7c3c7ccf49b8 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303088689dfe895bc41b1895f7c3c7ccf49b8 2023-03-08 18:56:22,343 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308040568f4a8c048c3b1e43b75b9767ed0 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308040568f4a8c048c3b1e43b75b9767ed0 2023-03-08 18:56:22,737 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/7396cf433f474722955d9663f876ee42 2023-03-08 18:56:22,740 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308836e261716f1419eb43f6db8154f7d6d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308836e261716f1419eb43f6db8154f7d6d 2023-03-08 18:56:22,741 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/b7eb2b55c5ee4ffa80349f34634cf1fe as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/b7eb2b55c5ee4ffa80349f34634cf1fe 2023-03-08 18:56:22,747 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/674ad1a8f0ba448eaa7ae12229ff21f5 2023-03-08 18:56:22,750 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/844a6729542349ca9abf79fc516e25d4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/844a6729542349ca9abf79fc516e25d4 2023-03-08 18:56:22,765 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/b7eb2b55c5ee4ffa80349f34634cf1fe, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 18:56:22,766 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/7396cf433f474722955d9663f876ee42 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/7396cf433f474722955d9663f876ee42 2023-03-08 18:56:22,768 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/7396cf433f474722955d9663f876ee42, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:56:22,769 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 7c4e5be8c7bf7634fdec5582cd8f9f5e in 1339ms, sequenceid=9, compaction requested=false 2023-03-08 18:56:22,769 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:22,769 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:22,769 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:56:22,784 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/844a6729542349ca9abf79fc516e25d4, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 18:56:22,785 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/674ad1a8f0ba448eaa7ae12229ff21f5 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/674ad1a8f0ba448eaa7ae12229ff21f5 2023-03-08 18:56:22,788 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/674ad1a8f0ba448eaa7ae12229ff21f5, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:56:22,789 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 8b7ffc3b5de37aaac329ae67ecaf3e9f in 1359ms, sequenceid=9, compaction requested=false 2023-03-08 18:56:22,789 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:22,789 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:22,789 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:56:23,145 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/c55a4a7eb4c548168c233db8fe8f7123 2023-03-08 18:56:23,149 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/4c2b4c0863f5498e86047411f5545b4f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/4c2b4c0863f5498e86047411f5545b4f 2023-03-08 18:56:23,172 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/4c2b4c0863f5498e86047411f5545b4f, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 18:56:23,173 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/c55a4a7eb4c548168c233db8fe8f7123 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c55a4a7eb4c548168c233db8fe8f7123 2023-03-08 18:56:23,177 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c55a4a7eb4c548168c233db8fe8f7123, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 18:56:23,177 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f3e546ab6e95b3c35d94bf0350799d5a in 1747ms, sequenceid=9, compaction requested=false 2023-03-08 18:56:23,177 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:23,177 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:23,177 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:56:23,178 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:56:23,178 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,178 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 18:56:23,178 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 18:56:23,180 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,180 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:23,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:23,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:23,181 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:23,181 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:23,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:23,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:23,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:23,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:23,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 18:56:23,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4523381d[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:23,182 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 18:56:23,182 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:23,183 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:23,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:23,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:23,184 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 18:56:23,184 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 18:56:23,184 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:23,184 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,184 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:23,185 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 18:56:23,185 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:23,185 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,185 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 18:56:23,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:23,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:23,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 18:56:23,188 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 18:56:23,188 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:23,188 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:23,188 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 18:56:23,188 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:23,189 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,189 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:23,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:23,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,189 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:23,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:23,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:23,190 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:23,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:23,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:23,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:23,191 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,194 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:23,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:23,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:23,194 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 18:56:23,194 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:23,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,194 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:23,195 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:23,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:23,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:23,295 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:23,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:23,296 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:23,296 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:23,297 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:23,298 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:23,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:23,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 18:56:23,300 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 18:56:23,300 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:23,300 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 18:56:23,300 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 18:56:23,301 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,301 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:23,302 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,302 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:23,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:23,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:23,302 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:23,302 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:23,303 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:23,303 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 18:56:23,303 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 18:56:23,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 18:56:23,303 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 18:56:23,303 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:23,303 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 18:56:23,304 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:23,304 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. started... 2023-03-08 18:56:23,304 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:56:23,304 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:23,304 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7c4e5be8c7bf7634fdec5582cd8f9f5e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:56:23,304 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:23,304 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. started... 2023-03-08 18:56:23,304 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:23,304 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. started... 2023-03-08 18:56:23,304 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f3e546ab6e95b3c35d94bf0350799d5a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:56:23,304 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8b7ffc3b5de37aaac329ae67ecaf3e9f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 18:56:23,361 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030880947256ad9643399f698f30c18354cb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030880947256ad9643399f698f30c18354cb 2023-03-08 18:56:23,361 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303089e341dde4f774e1db539330412ed9203 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303089e341dde4f774e1db539330412ed9203 2023-03-08 18:56:23,756 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308ef7cc2151dc446c2843db39e29a4eee2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308ef7cc2151dc446c2843db39e29a4eee2 2023-03-08 18:56:23,777 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/7865cff077314758bc8e13423b0e6a16 2023-03-08 18:56:23,784 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/cfc0712fabb344889e6323d6bec48356 2023-03-08 18:56:24,172 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/0ac8972da5db41fa80d04b1acf472a95 2023-03-08 18:56:24,223 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303086d28f44cc7e34f9f8a5aeaa3d7ec60c9 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303086d28f44cc7e34f9f8a5aeaa3d7ec60c9 2023-03-08 18:56:24,225 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085f18a804aad0411898bc45106b050224 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303085f18a804aad0411898bc45106b050224 2023-03-08 18:56:24,229 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/839cf3caf1c94dd6950d7a91ecc0ec59 2023-03-08 18:56:24,232 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/cfc0712fabb344889e6323d6bec48356 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/cfc0712fabb344889e6323d6bec48356 2023-03-08 18:56:24,255 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/cfc0712fabb344889e6323d6bec48356, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 18:56:24,256 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/839cf3caf1c94dd6950d7a91ecc0ec59 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/839cf3caf1c94dd6950d7a91ecc0ec59 2023-03-08 18:56:24,259 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/839cf3caf1c94dd6950d7a91ecc0ec59, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:56:24,260 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 8b7ffc3b5de37aaac329ae67ecaf3e9f in 955ms, sequenceid=13, compaction requested=true 2023-03-08 18:56:24,260 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:24,260 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:24,260 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:56:24,495 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 18:56:24,606 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308f67958505db2493abb76ccf06ad8c901 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308f67958505db2493abb76ccf06ad8c901 2023-03-08 18:56:24,610 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/4f8fa776f21f4f2081b22add76eab526 2023-03-08 18:56:24,613 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/0ac8972da5db41fa80d04b1acf472a95 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/0ac8972da5db41fa80d04b1acf472a95 2023-03-08 18:56:24,628 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/0c114ed3b1b54440b2555b46bdac2982 2023-03-08 18:56:24,632 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/7865cff077314758bc8e13423b0e6a16 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/7865cff077314758bc8e13423b0e6a16 2023-03-08 18:56:24,638 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/0ac8972da5db41fa80d04b1acf472a95, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 18:56:24,639 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/4f8fa776f21f4f2081b22add76eab526 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/4f8fa776f21f4f2081b22add76eab526 2023-03-08 18:56:24,644 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/4f8fa776f21f4f2081b22add76eab526, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:56:24,644 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 7c4e5be8c7bf7634fdec5582cd8f9f5e in 1340ms, sequenceid=13, compaction requested=true 2023-03-08 18:56:24,644 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:24,644 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:24,644 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:56:24,658 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/7865cff077314758bc8e13423b0e6a16, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 18:56:24,659 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/0c114ed3b1b54440b2555b46bdac2982 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/0c114ed3b1b54440b2555b46bdac2982 2023-03-08 18:56:24,662 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/0c114ed3b1b54440b2555b46bdac2982, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 18:56:24,663 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f3e546ab6e95b3c35d94bf0350799d5a in 1359ms, sequenceid=13, compaction requested=true 2023-03-08 18:56:24,663 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:24,663 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:24,663 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:56:24,663 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:56:24,663 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,663 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 18:56:24,663 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 18:56:24,665 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:24,665 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:24,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:24,665 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:24,665 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:24,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:24,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:24,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:24,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:24,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 18:56:24,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65dfe37a[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:24,667 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 18:56:24,667 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:24,668 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:24,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:24,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:24,668 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 18:56:24,668 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 18:56:24,668 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,668 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:24,668 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:24,670 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 18:56:24,670 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:24,670 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,670 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 18:56:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:24,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:24,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:24,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 18:56:24,672 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 18:56:24,672 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:24,672 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:24,672 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 18:56:24,672 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:24,674 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,674 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,674 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:24,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:24,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:24,675 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:24,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:24,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:24,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:24,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:24,676 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:24,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:24,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,678 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,678 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:24,678 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:24,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:24,678 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 18:56:24,678 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:24,678 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:24,678 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:24,678 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 18:56:24,679 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:24,679 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:24,678 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:24,678 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:24,679 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:24,679 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:24,679 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:24,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:24,779 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:24,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:24,781 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:24,781 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:24,781 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 18:56:24,783 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:24,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:24,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 18:56:24,785 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 18:56:24,785 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:24,785 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 18:56:24,785 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 18:56:24,785 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,785 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:24,790 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:24,790 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:24,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:24,790 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:24,790 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:24,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:24,791 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:24,791 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 18:56:24,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 18:56:24,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 18:56:24,791 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 18:56:24,791 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:24,791 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 18:56:24,792 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:24,792 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:56:24,792 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:24,792 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:24,792 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:24,792 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. started... 2023-03-08 18:56:24,792 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. started... 2023-03-08 18:56:24,792 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. started... 2023-03-08 18:56:24,792 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7c4e5be8c7bf7634fdec5582cd8f9f5e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:56:24,792 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8b7ffc3b5de37aaac329ae67ecaf3e9f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:56:24,792 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f3e546ab6e95b3c35d94bf0350799d5a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 18:56:24,841 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308be244362cebf48f2a6aeb35d869174f6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308be244362cebf48f2a6aeb35d869174f6 2023-03-08 18:56:25,236 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a60b0e87806c4e76a4930a08a480611a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a60b0e87806c4e76a4930a08a480611a 2023-03-08 18:56:25,241 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308d8324042001640e999c29186f1ca1905 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308d8324042001640e999c29186f1ca1905 2023-03-08 18:56:25,255 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/f190df6ec01d4aa1b9a96bdf9574410a 2023-03-08 18:56:25,652 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/9ed2b390a54a46dca6e9cb9f27490e60 2023-03-08 18:56:25,660 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/a98a9dd6fc4544338b54977a39d5a877 2023-03-08 18:56:25,687 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308ac958b53d0f942518be4f1573925c27d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308ac958b53d0f942518be4f1573925c27d 2023-03-08 18:56:26,087 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303085bfea3e4af3348f69c7bd6e671ae07e2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303085bfea3e4af3348f69c7bd6e671ae07e2 2023-03-08 18:56:26,096 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/f546c237951942609a2f57e03851093f 2023-03-08 18:56:26,099 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/f190df6ec01d4aa1b9a96bdf9574410a as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f190df6ec01d4aa1b9a96bdf9574410a 2023-03-08 18:56:26,107 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303084f94f416ea2742f69f4edc8a61b6d098 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303084f94f416ea2742f69f4edc8a61b6d098 2023-03-08 18:56:26,123 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f190df6ec01d4aa1b9a96bdf9574410a, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 18:56:26,123 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/f546c237951942609a2f57e03851093f as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/f546c237951942609a2f57e03851093f 2023-03-08 18:56:26,126 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/f546c237951942609a2f57e03851093f, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:56:26,127 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 7c4e5be8c7bf7634fdec5582cd8f9f5e in 1335ms, sequenceid=17, compaction requested=true 2023-03-08 18:56:26,127 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:26,127 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:26,127 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:56:26,493 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/54b125bd94ff447e94eb445d35ca92f3 2023-03-08 18:56:26,497 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/9ed2b390a54a46dca6e9cb9f27490e60 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/9ed2b390a54a46dca6e9cb9f27490e60 2023-03-08 18:56:26,511 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/357480bffba646a19fddb93eb7ee6bfe 2023-03-08 18:56:26,515 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/a98a9dd6fc4544338b54977a39d5a877 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/a98a9dd6fc4544338b54977a39d5a877 2023-03-08 18:56:26,520 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/9ed2b390a54a46dca6e9cb9f27490e60, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 18:56:26,521 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/54b125bd94ff447e94eb445d35ca92f3 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/54b125bd94ff447e94eb445d35ca92f3 2023-03-08 18:56:26,524 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/54b125bd94ff447e94eb445d35ca92f3, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:56:26,525 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f3e546ab6e95b3c35d94bf0350799d5a in 1733ms, sequenceid=17, compaction requested=true 2023-03-08 18:56:26,525 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:26,525 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:26,525 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:56:26,538 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/a98a9dd6fc4544338b54977a39d5a877, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 18:56:26,539 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/357480bffba646a19fddb93eb7ee6bfe as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/357480bffba646a19fddb93eb7ee6bfe 2023-03-08 18:56:26,542 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/357480bffba646a19fddb93eb7ee6bfe, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 18:56:26,543 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 8b7ffc3b5de37aaac329ae67ecaf3e9f in 1751ms, sequenceid=17, compaction requested=true 2023-03-08 18:56:26,543 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:26,543 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:26,543 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:56:26,543 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:56:26,543 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,543 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 18:56:26,543 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 18:56:26,545 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,545 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:26,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:26,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:26,545 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:26,545 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:26,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:26,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:26,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:26,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:26,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 18:56:26,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2036f58d[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:26,547 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 18:56:26,547 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:26,548 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:26,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:26,548 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 18:56:26,548 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 18:56:26,548 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:26,548 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,548 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:26,549 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 18:56:26,549 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,550 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:26,550 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 18:56:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:26,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:26,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:26,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:26,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 18:56:26,552 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 18:56:26,552 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:26,552 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:26,552 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 18:56:26,552 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:26,555 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,555 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:26,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,555 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:26,555 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:26,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:26,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:26,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:26,556 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:26,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:26,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,558 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,558 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:26,558 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:26,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:26,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:26,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 18:56:26,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:26,558 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:26,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 18:56:26,558 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:26,558 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:26,559 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:26,559 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:26,559 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:26,559 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:26,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:26,559 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:26,559 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,659 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:26,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:26,668 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:26,675 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:26,684 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:26,685 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:26,755 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:56:26,755 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20d7c20f to 127.0.0.1:54098 2023-03-08 18:56:26,756 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:26,756 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:56:26,756 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x61d18c9e to 127.0.0.1:54098 2023-03-08 18:56:26,756 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:26,792 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(933): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-03-08 18:56:26,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 18:56:26,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 18:56:26,793 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 18:56:26,793 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:26,793 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 18:56:26,793 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 18:56:26,794 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,794 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:26,796 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:26,796 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:26,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:26,796 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:26,796 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 18:56:26,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:26,796 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:26,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 18:56:26,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:26,797 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 18:56:26,797 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 18:56:26,797 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 18:56:26,797 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 18:56:26,797 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 18:56:26,797 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:26,797 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. started... 2023-03-08 18:56:26,797 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:26,797 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. started... 2023-03-08 18:56:26,797 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7c4e5be8c7bf7634fdec5582cd8f9f5e 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:56:26,797 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f3e546ab6e95b3c35d94bf0350799d5a 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:56:26,798 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 18:56:26,798 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 18:56:26,798 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:26,798 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. started... 2023-03-08 18:56:26,798 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8b7ffc3b5de37aaac329ae67ecaf3e9f 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 18:56:27,243 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/5a136bbbf5ab46b5bc54b59255974b47 2023-03-08 18:56:27,267 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a136bbbf5ab46b5bc54b59255974b47 2023-03-08 18:56:27,645 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/628bae64521a48a39d41a4f6ba58545c 2023-03-08 18:56:27,648 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/f57d039f111d4c458fbcff9874dde41d 2023-03-08 18:56:27,674 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 628bae64521a48a39d41a4f6ba58545c 2023-03-08 18:56:27,686 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f57d039f111d4c458fbcff9874dde41d 2023-03-08 18:56:28,083 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/6c0b8b93f55144c89eb6935e3bcd3ae4 2023-03-08 18:56:28,086 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c0b8b93f55144c89eb6935e3bcd3ae4 2023-03-08 18:56:28,086 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/5a136bbbf5ab46b5bc54b59255974b47 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/5a136bbbf5ab46b5bc54b59255974b47 2023-03-08 18:56:28,112 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a136bbbf5ab46b5bc54b59255974b47 2023-03-08 18:56:28,112 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/5a136bbbf5ab46b5bc54b59255974b47, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 18:56:28,112 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/6c0b8b93f55144c89eb6935e3bcd3ae4 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/6c0b8b93f55144c89eb6935e3bcd3ae4 2023-03-08 18:56:28,116 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c0b8b93f55144c89eb6935e3bcd3ae4 2023-03-08 18:56:28,116 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/6c0b8b93f55144c89eb6935e3bcd3ae4, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 18:56:28,116 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 8b7ffc3b5de37aaac329ae67ecaf3e9f in 1318ms, sequenceid=23, compaction requested=true 2023-03-08 18:56:28,117 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:28,117 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:28,117 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 18:56:28,501 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/c16c20667cd6481091932400c899067e 2023-03-08 18:56:28,504 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c16c20667cd6481091932400c899067e 2023-03-08 18:56:28,505 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/628bae64521a48a39d41a4f6ba58545c as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/628bae64521a48a39d41a4f6ba58545c 2023-03-08 18:56:28,509 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/b27281d7ee5247cb876ea49123dbe952 2023-03-08 18:56:28,512 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b27281d7ee5247cb876ea49123dbe952 2023-03-08 18:56:28,512 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/f57d039f111d4c458fbcff9874dde41d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f57d039f111d4c458fbcff9874dde41d 2023-03-08 18:56:28,531 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 628bae64521a48a39d41a4f6ba58545c 2023-03-08 18:56:28,531 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/628bae64521a48a39d41a4f6ba58545c, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 18:56:28,532 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/c16c20667cd6481091932400c899067e as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c16c20667cd6481091932400c899067e 2023-03-08 18:56:28,536 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c16c20667cd6481091932400c899067e 2023-03-08 18:56:28,536 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f57d039f111d4c458fbcff9874dde41d 2023-03-08 18:56:28,536 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c16c20667cd6481091932400c899067e, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 18:56:28,536 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f57d039f111d4c458fbcff9874dde41d, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 18:56:28,537 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/b27281d7ee5247cb876ea49123dbe952 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/b27281d7ee5247cb876ea49123dbe952 2023-03-08 18:56:28,537 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f3e546ab6e95b3c35d94bf0350799d5a in 1740ms, sequenceid=23, compaction requested=true 2023-03-08 18:56:28,537 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:28,537 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:28,537 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 18:56:28,540 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b27281d7ee5247cb876ea49123dbe952 2023-03-08 18:56:28,540 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/b27281d7ee5247cb876ea49123dbe952, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 18:56:28,541 INFO [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 7c4e5be8c7bf7634fdec5582cd8f9f5e in 1744ms, sequenceid=23, compaction requested=true 2023-03-08 18:56:28,541 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:28,541 DEBUG [rs(jenkins-hbase6.apache.org,46397,1678301621627)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:28,541 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 18:56:28,541 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 18:56:28,541 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,541 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 18:56:28,541 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 18:56:28,543 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,543 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:28,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:28,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:28,543 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:28,543 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 18:56:28,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:28,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,46397,1678301621627' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 18:56:28,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a9be995[Count = 0] remaining members to acquire global barrier 2023-03-08 18:56:28,545 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 18:56:28,545 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:28,546 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:28,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:28,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:28,546 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 18:56:28,546 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 18:56:28,546 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase6.apache.org,46397,1678301621627' in zk 2023-03-08 18:56:28,550 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,551 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 18:56:28,552 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 18:56:28,552 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:28,552 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,552 DEBUG [member: 'jenkins-hbase6.apache.org,46397,1678301621627' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 18:56:28,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:28,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:28,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:28,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:28,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:28,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:28,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:28,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase6.apache.org,46397,1678301621627': 2023-03-08 18:56:28,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,46397,1678301621627' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 18:56:28,554 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 18:56:28,554 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 18:56:28,554 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 18:56:28,554 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 18:56:28,554 INFO [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 18:56:28,557 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:28,557 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:28,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:28,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:28,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 18:56:28,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 18:56:28,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:28,557 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:28,557 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 18:56:28,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:28,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:28,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:28,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:28,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 18:56:28,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:28,558 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 18:56:28,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 18:56:28,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,560 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:28,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 18:56:28,561 DEBUG [(jenkins-hbase6.apache.org,44349,1678301619149)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 18:56:28,561 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 18:56:28,561 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 18:56:28,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 18:56:28,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 18:56:28,561 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 18:56:28,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 18:56:28,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 18:56:28,662 DEBUG [Listener at localhost/36281] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 18:56:28,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44349] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 18:56:28,663 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:56:28,663 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 is initiating major compaction (all files) 2023-03-08 18:56:28,663 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:56:28,663 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:56:28,663 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 in testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:28,663 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/2480abcb54af474ba2e0c5955c42f05a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/b7eb2b55c5ee4ffa80349f34634cf1fe, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/0ac8972da5db41fa80d04b1acf472a95, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f190df6ec01d4aa1b9a96bdf9574410a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f57d039f111d4c458fbcff9874dde41d] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp, totalSize=27.1 K 2023-03-08 18:56:28,664 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 2480abcb54af474ba2e0c5955c42f05a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301779930 2023-03-08 18:56:28,664 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting b7eb2b55c5ee4ffa80349f34634cf1fe, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301781422 2023-03-08 18:56:28,665 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 0ac8972da5db41fa80d04b1acf472a95, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301783296 2023-03-08 18:56:28,665 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting f190df6ec01d4aa1b9a96bdf9574410a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301784781 2023-03-08 18:56:28,666 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting f57d039f111d4c458fbcff9874dde41d, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:56:28,676 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 7c4e5be8c7bf7634fdec5582cd8f9f5e#family1#compaction#198 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:29,116 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030859c3ac34d2494aaabf61431bab0c90da_del 2023-03-08 18:56:29,116 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030859c3ac34d2494aaabf61431bab0c90da_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030859c3ac34d2494aaabf61431bab0c90da_del 2023-03-08 18:56:29,575 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00937bc39b2d43ad9dffbd1c471eca33 2023-03-08 18:56:29,576 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family1/00937bc39b2d43ad9dffbd1c471eca33 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/00937bc39b2d43ad9dffbd1c471eca33 2023-03-08 18:56:29,600 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00937bc39b2d43ad9dffbd1c471eca33 2023-03-08 18:56:29,601 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 of 7c4e5be8c7bf7634fdec5582cd8f9f5e into 00937bc39b2d43ad9dffbd1c471eca33(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:56:29,601 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:29,601 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:56:29,601 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 7c4e5be8c7bf7634fdec5582cd8f9f5e/family2 is initiating major compaction (all files) 2023-03-08 18:56:29,601 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:56:29,601 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:56:29,603 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 7c4e5be8c7bf7634fdec5582cd8f9f5e/family2 in testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:29,603 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/faa2f88a3b4a4a6dbc8a9885ca894a82, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/7396cf433f474722955d9663f876ee42, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/4f8fa776f21f4f2081b22add76eab526, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/f546c237951942609a2f57e03851093f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/b27281d7ee5247cb876ea49123dbe952] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp, totalSize=25.4 K 2023-03-08 18:56:29,604 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting faa2f88a3b4a4a6dbc8a9885ca894a82, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301779930 2023-03-08 18:56:29,604 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 7396cf433f474722955d9663f876ee42, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301781422 2023-03-08 18:56:29,605 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 4f8fa776f21f4f2081b22add76eab526, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301783296 2023-03-08 18:56:29,605 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting f546c237951942609a2f57e03851093f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301784781 2023-03-08 18:56:29,605 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting b27281d7ee5247cb876ea49123dbe952, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:56:29,614 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 7c4e5be8c7bf7634fdec5582cd8f9f5e#family2#compaction#199 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:29,624 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030812f694223193402d923051876a1a80ad_del 2023-03-08 18:56:29,625 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030812f694223193402d923051876a1a80ad_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023030812f694223193402d923051876a1a80ad_del 2023-03-08 18:56:29,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/2480abcb54af474ba2e0c5955c42f05a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/b7eb2b55c5ee4ffa80349f34634cf1fe, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/0ac8972da5db41fa80d04b1acf472a95, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f190df6ec01d4aa1b9a96bdf9574410a, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f57d039f111d4c458fbcff9874dde41d] to archive 2023-03-08 18:56:29,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:29,691 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/2480abcb54af474ba2e0c5955c42f05a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/2480abcb54af474ba2e0c5955c42f05a 2023-03-08 18:56:29,692 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/b7eb2b55c5ee4ffa80349f34634cf1fe to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/b7eb2b55c5ee4ffa80349f34634cf1fe 2023-03-08 18:56:29,694 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/0ac8972da5db41fa80d04b1acf472a95 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/0ac8972da5db41fa80d04b1acf472a95 2023-03-08 18:56:29,695 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f190df6ec01d4aa1b9a96bdf9574410a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f190df6ec01d4aa1b9a96bdf9574410a 2023-03-08 18:56:29,696 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f57d039f111d4c458fbcff9874dde41d to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/f57d039f111d4c458fbcff9874dde41d 2023-03-08 18:56:30,451 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08143e7fe8004c889e8c5d8db8f9cf8c 2023-03-08 18:56:30,452 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/.tmp/family2/08143e7fe8004c889e8c5d8db8f9cf8c as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/08143e7fe8004c889e8c5d8db8f9cf8c 2023-03-08 18:56:30,455 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08143e7fe8004c889e8c5d8db8f9cf8c 2023-03-08 18:56:30,456 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7c4e5be8c7bf7634fdec5582cd8f9f5e/family2 of 7c4e5be8c7bf7634fdec5582cd8f9f5e into 08143e7fe8004c889e8c5d8db8f9cf8c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:56:30,456 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:30,456 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:56:30,456 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): f3e546ab6e95b3c35d94bf0350799d5a/family1 is initiating major compaction (all files) 2023-03-08 18:56:30,457 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:56:30,457 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:56:30,457 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of f3e546ab6e95b3c35d94bf0350799d5a/family1 in testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:30,457 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/cac306320bab4be9b764764628b3a028, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/4c2b4c0863f5498e86047411f5545b4f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/7865cff077314758bc8e13423b0e6a16, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/9ed2b390a54a46dca6e9cb9f27490e60, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/628bae64521a48a39d41a4f6ba58545c] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp, totalSize=27.1 K 2023-03-08 18:56:30,457 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting cac306320bab4be9b764764628b3a028, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301779930 2023-03-08 18:56:30,458 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 4c2b4c0863f5498e86047411f5545b4f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301781421 2023-03-08 18:56:30,459 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 7865cff077314758bc8e13423b0e6a16, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301783296 2023-03-08 18:56:30,459 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 9ed2b390a54a46dca6e9cb9f27490e60, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301784781 2023-03-08 18:56:30,460 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 628bae64521a48a39d41a4f6ba58545c, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:56:30,468 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): f3e546ab6e95b3c35d94bf0350799d5a#family1#compaction#200 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:30,483 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/faa2f88a3b4a4a6dbc8a9885ca894a82, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/7396cf433f474722955d9663f876ee42, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/4f8fa776f21f4f2081b22add76eab526, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/f546c237951942609a2f57e03851093f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/b27281d7ee5247cb876ea49123dbe952] to archive 2023-03-08 18:56:30,483 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:30,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/faa2f88a3b4a4a6dbc8a9885ca894a82 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/faa2f88a3b4a4a6dbc8a9885ca894a82 2023-03-08 18:56:30,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/7396cf433f474722955d9663f876ee42 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/7396cf433f474722955d9663f876ee42 2023-03-08 18:56:30,488 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/4f8fa776f21f4f2081b22add76eab526 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/4f8fa776f21f4f2081b22add76eab526 2023-03-08 18:56:30,489 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/f546c237951942609a2f57e03851093f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/f546c237951942609a2f57e03851093f 2023-03-08 18:56:30,490 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/b27281d7ee5247cb876ea49123dbe952 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family2/b27281d7ee5247cb876ea49123dbe952 2023-03-08 18:56:30,511 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303080e8fa1f5b1514c8b9127ee133a01eaf5_del 2023-03-08 18:56:30,511 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303080e8fa1f5b1514c8b9127ee133a01eaf5_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303080e8fa1f5b1514c8b9127ee133a01eaf5_del 2023-03-08 18:56:31,369 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4bf96334960a401f8ef7f4f4f4813b43 2023-03-08 18:56:31,370 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family1/4bf96334960a401f8ef7f4f4f4813b43 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/4bf96334960a401f8ef7f4f4f4813b43 2023-03-08 18:56:31,392 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4bf96334960a401f8ef7f4f4f4813b43 2023-03-08 18:56:31,393 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f3e546ab6e95b3c35d94bf0350799d5a/family1 of f3e546ab6e95b3c35d94bf0350799d5a into 4bf96334960a401f8ef7f4f4f4813b43(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:56:31,393 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:31,394 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:56:31,394 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): f3e546ab6e95b3c35d94bf0350799d5a/family2 is initiating major compaction (all files) 2023-03-08 18:56:31,394 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:56:31,394 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:56:31,394 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of f3e546ab6e95b3c35d94bf0350799d5a/family2 in testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:31,394 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/f267406f52a24517a856e7dda045db47, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c55a4a7eb4c548168c233db8fe8f7123, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/0c114ed3b1b54440b2555b46bdac2982, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/54b125bd94ff447e94eb445d35ca92f3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c16c20667cd6481091932400c899067e] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp, totalSize=25.4 K 2023-03-08 18:56:31,394 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting f267406f52a24517a856e7dda045db47, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301779930 2023-03-08 18:56:31,395 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting c55a4a7eb4c548168c233db8fe8f7123, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301781421 2023-03-08 18:56:31,395 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 0c114ed3b1b54440b2555b46bdac2982, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301783296 2023-03-08 18:56:31,396 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 54b125bd94ff447e94eb445d35ca92f3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301784781 2023-03-08 18:56:31,396 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting c16c20667cd6481091932400c899067e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:56:31,404 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): f3e546ab6e95b3c35d94bf0350799d5a#family2#compaction#201 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:31,412 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308053525efdf7943ac89a8fe63d32703aa_del 2023-03-08 18:56:31,412 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308053525efdf7943ac89a8fe63d32703aa_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308053525efdf7943ac89a8fe63d32703aa_del 2023-03-08 18:56:31,422 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9053e236996e47549f9aa2ac351f2289 2023-03-08 18:56:31,423 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/.tmp/family2/9053e236996e47549f9aa2ac351f2289 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/9053e236996e47549f9aa2ac351f2289 2023-03-08 18:56:31,425 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9053e236996e47549f9aa2ac351f2289 2023-03-08 18:56:31,426 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f3e546ab6e95b3c35d94bf0350799d5a/family2 of f3e546ab6e95b3c35d94bf0350799d5a into 9053e236996e47549f9aa2ac351f2289(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:56:31,426 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:31,426 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:56:31,426 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 is initiating major compaction (all files) 2023-03-08 18:56:31,426 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:56:31,426 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:56:31,426 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 in testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:31,426 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/0a76e77c02b346c6977cdefc27112a9b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/844a6729542349ca9abf79fc516e25d4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/cfc0712fabb344889e6323d6bec48356, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/a98a9dd6fc4544338b54977a39d5a877, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/5a136bbbf5ab46b5bc54b59255974b47] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp, totalSize=27.1 K 2023-03-08 18:56:31,427 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 0a76e77c02b346c6977cdefc27112a9b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301779931 2023-03-08 18:56:31,427 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 844a6729542349ca9abf79fc516e25d4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301781422 2023-03-08 18:56:31,427 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting cfc0712fabb344889e6323d6bec48356, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301783297 2023-03-08 18:56:31,427 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting a98a9dd6fc4544338b54977a39d5a877, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301784781 2023-03-08 18:56:31,428 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 5a136bbbf5ab46b5bc54b59255974b47, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:56:31,435 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 8b7ffc3b5de37aaac329ae67ecaf3e9f#family1#compaction#202 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:31,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/cac306320bab4be9b764764628b3a028, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/4c2b4c0863f5498e86047411f5545b4f, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/7865cff077314758bc8e13423b0e6a16, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/9ed2b390a54a46dca6e9cb9f27490e60, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/628bae64521a48a39d41a4f6ba58545c] to archive 2023-03-08 18:56:31,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:31,483 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/cac306320bab4be9b764764628b3a028 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/cac306320bab4be9b764764628b3a028 2023-03-08 18:56:31,484 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/4c2b4c0863f5498e86047411f5545b4f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/4c2b4c0863f5498e86047411f5545b4f 2023-03-08 18:56:31,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/7865cff077314758bc8e13423b0e6a16 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/7865cff077314758bc8e13423b0e6a16 2023-03-08 18:56:31,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/9ed2b390a54a46dca6e9cb9f27490e60 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/9ed2b390a54a46dca6e9cb9f27490e60 2023-03-08 18:56:31,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/628bae64521a48a39d41a4f6ba58545c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/628bae64521a48a39d41a4f6ba58545c 2023-03-08 18:56:31,487 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/f267406f52a24517a856e7dda045db47, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c55a4a7eb4c548168c233db8fe8f7123, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/0c114ed3b1b54440b2555b46bdac2982, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/54b125bd94ff447e94eb445d35ca92f3, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c16c20667cd6481091932400c899067e] to archive 2023-03-08 18:56:31,488 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:31,489 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/f267406f52a24517a856e7dda045db47 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/f267406f52a24517a856e7dda045db47 2023-03-08 18:56:31,490 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c55a4a7eb4c548168c233db8fe8f7123 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c55a4a7eb4c548168c233db8fe8f7123 2023-03-08 18:56:31,490 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/0c114ed3b1b54440b2555b46bdac2982 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/0c114ed3b1b54440b2555b46bdac2982 2023-03-08 18:56:31,491 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/54b125bd94ff447e94eb445d35ca92f3 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/54b125bd94ff447e94eb445d35ca92f3 2023-03-08 18:56:31,492 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c16c20667cd6481091932400c899067e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family2/c16c20667cd6481091932400c899067e 2023-03-08 18:56:31,879 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303089e1f63bc55194f1a823abb44e31d3c88_del 2023-03-08 18:56:31,880 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303089e1f63bc55194f1a823abb44e31d3c88_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303089e1f63bc55194f1a823abb44e31d3c88_del 2023-03-08 18:56:31,942 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a3454b04002b43449b61d977b3b022b2 2023-03-08 18:56:31,943 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family1/a3454b04002b43449b61d977b3b022b2 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/a3454b04002b43449b61d977b3b022b2 2023-03-08 18:56:31,965 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a3454b04002b43449b61d977b3b022b2 2023-03-08 18:56:31,966 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 of 8b7ffc3b5de37aaac329ae67ecaf3e9f into a3454b04002b43449b61d977b3b022b2(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:56:31,966 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:31,966 DEBUG [Listener at localhost/36281] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 18:56:31,966 DEBUG [Listener at localhost/36281] regionserver.HStore(1912): 8b7ffc3b5de37aaac329ae67ecaf3e9f/family2 is initiating major compaction (all files) 2023-03-08 18:56:31,966 INFO [Listener at localhost/36281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 18:56:31,967 INFO [Listener at localhost/36281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 18:56:31,967 INFO [Listener at localhost/36281] regionserver.HRegion(2259): Starting compaction of 8b7ffc3b5de37aaac329ae67ecaf3e9f/family2 in testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:31,967 INFO [Listener at localhost/36281] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/d941504510614fdfa609225803ca5ce2, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/674ad1a8f0ba448eaa7ae12229ff21f5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/839cf3caf1c94dd6950d7a91ecc0ec59, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/357480bffba646a19fddb93eb7ee6bfe, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/6c0b8b93f55144c89eb6935e3bcd3ae4] into tmpdir=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp, totalSize=25.4 K 2023-03-08 18:56:31,967 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting d941504510614fdfa609225803ca5ce2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678301779931 2023-03-08 18:56:31,967 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 674ad1a8f0ba448eaa7ae12229ff21f5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678301781422 2023-03-08 18:56:31,968 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 839cf3caf1c94dd6950d7a91ecc0ec59, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678301783297 2023-03-08 18:56:31,968 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 357480bffba646a19fddb93eb7ee6bfe, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678301784781 2023-03-08 18:56:31,968 DEBUG [Listener at localhost/36281] compactions.Compactor(207): Compacting 6c0b8b93f55144c89eb6935e3bcd3ae4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 18:56:31,978 INFO [Listener at localhost/36281] throttle.PressureAwareThroughputController(145): 8b7ffc3b5de37aaac329ae67ecaf3e9f#family2#compaction#203 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 18:56:31,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/0a76e77c02b346c6977cdefc27112a9b, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/844a6729542349ca9abf79fc516e25d4, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/cfc0712fabb344889e6323d6bec48356, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/a98a9dd6fc4544338b54977a39d5a877, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/5a136bbbf5ab46b5bc54b59255974b47] to archive 2023-03-08 18:56:31,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:31,983 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/0a76e77c02b346c6977cdefc27112a9b to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/0a76e77c02b346c6977cdefc27112a9b 2023-03-08 18:56:31,984 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/844a6729542349ca9abf79fc516e25d4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/844a6729542349ca9abf79fc516e25d4 2023-03-08 18:56:31,985 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/cfc0712fabb344889e6323d6bec48356 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/cfc0712fabb344889e6323d6bec48356 2023-03-08 18:56:31,985 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030832662544e87841d6bf160908dc5ccc29_del 2023-03-08 18:56:31,986 INFO [Listener at localhost/36281] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030832662544e87841d6bf160908dc5ccc29_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023030832662544e87841d6bf160908dc5ccc29_del 2023-03-08 18:56:31,987 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/a98a9dd6fc4544338b54977a39d5a877 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/a98a9dd6fc4544338b54977a39d5a877 2023-03-08 18:56:31,988 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/5a136bbbf5ab46b5bc54b59255974b47 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/5a136bbbf5ab46b5bc54b59255974b47 2023-03-08 18:56:31,995 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3e26849d3eb74beaa6c0185ad2526b5e 2023-03-08 18:56:31,996 DEBUG [Listener at localhost/36281] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/.tmp/family2/3e26849d3eb74beaa6c0185ad2526b5e as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/3e26849d3eb74beaa6c0185ad2526b5e 2023-03-08 18:56:31,999 INFO [Listener at localhost/36281] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3e26849d3eb74beaa6c0185ad2526b5e 2023-03-08 18:56:32,000 INFO [Listener at localhost/36281] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8b7ffc3b5de37aaac329ae67ecaf3e9f/family2 of 8b7ffc3b5de37aaac329ae67ecaf3e9f into 3e26849d3eb74beaa6c0185ad2526b5e(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 18:56:32,000 DEBUG [Listener at localhost/36281] regionserver.HRegion(2289): Compaction status journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:32,003 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:32,006 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:32,009 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:32,010 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:32,022 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44349] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10#\x9B\x1C\x12\x95T\xC8\xF7\xC5\x91\x84\x08\x18\xF0\xE2\xD4"\x10\xA5\x19\xA8(\xDC\xEB\x94~O\xBD\xFAz\xEF\xE6\xD2\x9E*\x10=\x9DQ \x81\x1AR\xE8\xCD\xCEff\x8C\x5CWG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-08 18:56:32,053 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 18:56:32,054 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 18:56:32,054 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:56:32,054 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678302392054 2023-03-08 18:56:32,055 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 18:56:32,087 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/d941504510614fdfa609225803ca5ce2, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/674ad1a8f0ba448eaa7ae12229ff21f5, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/839cf3caf1c94dd6950d7a91ecc0ec59, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/357480bffba646a19fddb93eb7ee6bfe, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/6c0b8b93f55144c89eb6935e3bcd3ae4] to archive 2023-03-08 18:56:32,087 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:32,089 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/d941504510614fdfa609225803ca5ce2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/d941504510614fdfa609225803ca5ce2 2023-03-08 18:56:32,091 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/674ad1a8f0ba448eaa7ae12229ff21f5 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/674ad1a8f0ba448eaa7ae12229ff21f5 2023-03-08 18:56:32,092 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/839cf3caf1c94dd6950d7a91ecc0ec59 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/839cf3caf1c94dd6950d7a91ecc0ec59 2023-03-08 18:56:32,093 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/357480bffba646a19fddb93eb7ee6bfe to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/357480bffba646a19fddb93eb7ee6bfe 2023-03-08 18:56:32,095 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/6c0b8b93f55144c89eb6935e3bcd3ae4 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family2/6c0b8b93f55144c89eb6935e3bcd3ae4 2023-03-08 18:56:32,335 DEBUG [regionserver/jenkins-hbase6:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 18:56:32,420 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-08 18:56:32,442 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303080e8fa1f5b1514c8b9127ee133a01eaf5_del 2023-03-08 18:56:32,464 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030859c3ac34d2494aaabf61431bab0c90da_del 2023-03-08 18:56:32,488 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303089e1f63bc55194f1a823abb44e31d3c88_del 2023-03-08 18:56:32,488 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 18:56:32,489 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] zookeeper.ReadOnlyZKClient(139): Connect 0x7584e558 to 127.0.0.1:54098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 18:56:32,494 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@bca6dc1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 18:56:32,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 18:56:33,092 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303083f9cfffac6a74df7ac1f8953305664f0 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303083f9cfffac6a74df7ac1f8953305664f0 2023-03-08 18:56:33,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 18:56:33,105 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 18:56:33,107 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:57546, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 18:56:33,115 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 18:56:33,116 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:50708, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 18:56:33,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/17e4f6cf1e34452ebb80f2f832c16084 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-08 18:56:33,149 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/17e4f6cf1e34452ebb80f2f832c16084}] 2023-03-08 18:56:33,154 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/17e4f6cf1e34452ebb80f2f832c16084 for inclusion in 8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 2023-03-08 18:56:33,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-08 18:56:33,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 18:56:33,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:33,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/17e4f6cf1e34452ebb80f2f832c16084 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__tjr1i7j975fdp373ls4asfslmpl37jm7abn78o6oilhrtjums6m7ea9n35d4mio3/family1/17e4f6cf1e34452ebb80f2f832c16084 2023-03-08 18:56:33,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__tjr1i7j975fdp373ls4asfslmpl37jm7abn78o6oilhrtjums6m7ea9n35d4mio3/family1/17e4f6cf1e34452ebb80f2f832c16084 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/fc0c4ee65d90458f9ced87beaf1dc59c_SeqId_28_ 2023-03-08 18:56:33,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__tjr1i7j975fdp373ls4asfslmpl37jm7abn78o6oilhrtjums6m7ea9n35d4mio3/family1/17e4f6cf1e34452ebb80f2f832c16084 into 8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/fc0c4ee65d90458f9ced87beaf1dc59c_SeqId_28_ - updating store file list. 2023-03-08 18:56:33,199 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/fc0c4ee65d90458f9ced87beaf1dc59c_SeqId_28_ into 8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 2023-03-08 18:56:33,199 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__tjr1i7j975fdp373ls4asfslmpl37jm7abn78o6oilhrtjums6m7ea9n35d4mio3/family1/17e4f6cf1e34452ebb80f2f832c16084 into 8b7ffc3b5de37aaac329ae67ecaf3e9f/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/family1/fc0c4ee65d90458f9ced87beaf1dc59c_SeqId_28_) 2023-03-08 18:56:33,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__tjr1i7j975fdp373ls4asfslmpl37jm7abn78o6oilhrtjums6m7ea9n35d4mio3/family1/17e4f6cf1e34452ebb80f2f832c16084 2023-03-08 18:56:33,201 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 18:56:33,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:33,202 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303087a84c57fb6d24aed881cbe27076f625f to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303087a84c57fb6d24aed881cbe27076f625f 2023-03-08 18:56:33,203 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030880947256ad9643399f698f30c18354cb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030880947256ad9643399f698f30c18354cb 2023-03-08 18:56:33,204 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088d5b9d829f704e52b6a0c3afd234ac5e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088d5b9d829f704e52b6a0c3afd234ac5e 2023-03-08 18:56:33,205 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308d8324042001640e999c29186f1ca1905 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308d8324042001640e999c29186f1ca1905 2023-03-08 18:56:33,205 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 18:56:33,205 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 18:56:33,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 18:56:33,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 18:56:33,578 DEBUG [regionserver/jenkins-hbase6:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 18:56:34,159 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030879505b7d3d394c5fa1ae527202cee6cd to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030879505b7d3d394c5fa1ae527202cee6cd 2023-03-08 18:56:34,160 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 18:56:34,206 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/3b5d304a55b241bc941025be7c3f725c first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-08 18:56:34,207 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/3b5d304a55b241bc941025be7c3f725c}] 2023-03-08 18:56:34,209 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/3b5d304a55b241bc941025be7c3f725c for inclusion in f3e546ab6e95b3c35d94bf0350799d5a/family1 2023-03-08 18:56:34,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-08 18:56:34,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 18:56:34,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:34,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/3b5d304a55b241bc941025be7c3f725c to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__m1drm8nhjo8sm2jel8sh3o4atdj69g47c499btepve8ste1lg1cv18t5jk8evaur/family1/3b5d304a55b241bc941025be7c3f725c 2023-03-08 18:56:34,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__m1drm8nhjo8sm2jel8sh3o4atdj69g47c499btepve8ste1lg1cv18t5jk8evaur/family1/3b5d304a55b241bc941025be7c3f725c as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/feed3b9f71314906a08931cb41560dce_SeqId_28_ 2023-03-08 18:56:34,234 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__m1drm8nhjo8sm2jel8sh3o4atdj69g47c499btepve8ste1lg1cv18t5jk8evaur/family1/3b5d304a55b241bc941025be7c3f725c into f3e546ab6e95b3c35d94bf0350799d5a/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/feed3b9f71314906a08931cb41560dce_SeqId_28_ - updating store file list. 2023-03-08 18:56:34,255 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/feed3b9f71314906a08931cb41560dce_SeqId_28_ into f3e546ab6e95b3c35d94bf0350799d5a/family1 2023-03-08 18:56:34,255 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__m1drm8nhjo8sm2jel8sh3o4atdj69g47c499btepve8ste1lg1cv18t5jk8evaur/family1/3b5d304a55b241bc941025be7c3f725c into f3e546ab6e95b3c35d94bf0350799d5a/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/family1/feed3b9f71314906a08931cb41560dce_SeqId_28_) 2023-03-08 18:56:34,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__m1drm8nhjo8sm2jel8sh3o4atdj69g47c499btepve8ste1lg1cv18t5jk8evaur/family1/3b5d304a55b241bc941025be7c3f725c 2023-03-08 18:56:34,257 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 18:56:34,257 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:34,258 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030878c6c7c73b0544499d10ba392a7eeb06 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030878c6c7c73b0544499d10ba392a7eeb06 2023-03-08 18:56:34,259 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303089e341dde4f774e1db539330412ed9203 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303089e341dde4f774e1db539330412ed9203 2023-03-08 18:56:34,260 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a60b0e87806c4e76a4930a08a480611a to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a60b0e87806c4e76a4930a08a480611a 2023-03-08 18:56:34,261 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308c3d95e05e42540968bb8295e34877224 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308c3d95e05e42540968bb8295e34877224 2023-03-08 18:56:34,261 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 18:56:34,261 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 18:56:34,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 18:56:34,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 18:56:34,814 DEBUG [regionserver/jenkins-hbase6:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 18:56:35,217 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303081874ff2014ae41a48a426eb1006c73a6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303081874ff2014ae41a48a426eb1006c73a6 2023-03-08 18:56:35,218 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 18:56:35,263 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/2fd3143f38964e8898b47ef96e1bdbdb first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-08 18:56:35,266 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e., hostname=jenkins-hbase6.apache.org,46397,1678301621627, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/2fd3143f38964e8898b47ef96e1bdbdb}] 2023-03-08 18:56:35,268 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(763): Validating hfile at hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/2fd3143f38964e8898b47ef96e1bdbdb for inclusion in 7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 2023-03-08 18:56:35,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-08 18:56:35,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 18:56:35,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegion(2446): Flush status journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:35,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/2fd3143f38964e8898b47ef96e1bdbdb to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__bfis0evl64hrv95t30umte4e6bahgkjeb68s7qrscpv7roreeb8mmpgj4paq9r4k/family1/2fd3143f38964e8898b47ef96e1bdbdb 2023-03-08 18:56:35,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__bfis0evl64hrv95t30umte4e6bahgkjeb68s7qrscpv7roreeb8mmpgj4paq9r4k/family1/2fd3143f38964e8898b47ef96e1bdbdb as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/dccdbf5c4cec4d90907d930f2a5c963c_SeqId_28_ 2023-03-08 18:56:35,293 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(849): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__bfis0evl64hrv95t30umte4e6bahgkjeb68s7qrscpv7roreeb8mmpgj4paq9r4k/family1/2fd3143f38964e8898b47ef96e1bdbdb into 7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/dccdbf5c4cec4d90907d930f2a5c963c_SeqId_28_ - updating store file list. 2023-03-08 18:56:35,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(882): Loaded HFile hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/dccdbf5c4cec4d90907d930f2a5c963c_SeqId_28_ into 7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 2023-03-08 18:56:35,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.HStore(855): Successfully loaded hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__bfis0evl64hrv95t30umte4e6bahgkjeb68s7qrscpv7roreeb8mmpgj4paq9r4k/family1/2fd3143f38964e8898b47ef96e1bdbdb into 7c4e5be8c7bf7634fdec5582cd8f9f5e/family1 (new location: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/family1/dccdbf5c4cec4d90907d930f2a5c963c_SeqId_28_) 2023-03-08 18:56:35,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46397] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/staging/jenkins__testMajorCompactionFromAdmin__bfis0evl64hrv95t30umte4e6bahgkjeb68s7qrscpv7roreeb8mmpgj4paq9r4k/family1/2fd3143f38964e8898b47ef96e1bdbdb 2023-03-08 18:56:35,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 18:56:35,319 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:35,321 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303082a7935cb8d2d4d20a0ffc063894dba71 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303082a7935cb8d2d4d20a0ffc063894dba71 2023-03-08 18:56:35,322 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308b419575fda624d3888a087fbe827473e to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308b419575fda624d3888a087fbe827473e 2023-03-08 18:56:35,323 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308be244362cebf48f2a6aeb35d869174f6 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308be244362cebf48f2a6aeb35d869174f6 2023-03-08 18:56:35,324 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308ef7cc2151dc446c2843db39e29a4eee2 to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308ef7cc2151dc446c2843db39e29a4eee2 2023-03-08 18:56:35,324 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 18:56:35,324 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 18:56:35,325 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 18:56:35,325 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 18:56:35,325 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 18:56:35,325 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(367): [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303080e8fa1f5b1514c8b9127ee133a01eaf5_del] 2023-03-08 18:56:35,325 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:35,327 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303080e8fa1f5b1514c8b9127ee133a01eaf5_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303080e8fa1f5b1514c8b9127ee133a01eaf5_del 2023-03-08 18:56:35,327 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 18:56:35,327 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(367): [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030859c3ac34d2494aaabf61431bab0c90da_del] 2023-03-08 18:56:35,327 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:35,328 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030859c3ac34d2494aaabf61431bab0c90da_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030859c3ac34d2494aaabf61431bab0c90da_del 2023-03-08 18:56:35,328 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 18:56:35,328 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] compactions.PartitionedMobCompactor(367): [hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303089e1f63bc55194f1a823abb44e31d3c88_del] 2023-03-08 18:56:35,328 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 18:56:35,329 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303089e1f63bc55194f1a823abb44e31d3c88_del to hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303089e1f63bc55194f1a823abb44e31d3c88_del 2023-03-08 18:56:35,330 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 18:56:35,330 INFO [master/jenkins-hbase6:0:becomeActiveMaster-MasterMobCompaction-1678301625850] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 18:56:35,331 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 18:56:35,332 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.2790 sec 2023-03-08 18:56:35,348 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:35,350 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:35,352 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 18:56:35,353 DEBUG [Listener at localhost/36281] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 18:56:35,357 INFO [Listener at localhost/36281] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 18:56:35,398 INFO [Listener at localhost/36281] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=249 (was 257), OpenFileDescriptor=638 (was 645), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=198 (was 196) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6911 (was 6877) - AvailableMemoryMB LEAK? - 2023-03-08 18:56:35,399 DEBUG [Listener at localhost/36281] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33fc1e18 to 127.0.0.1:54098 2023-03-08 18:56:35,399 DEBUG [Listener at localhost/36281] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:35,399 INFO [Listener at localhost/36281] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-08 18:56:35,399 INFO [Listener at localhost/36281] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 18:56:35,399 DEBUG [Listener at localhost/36281] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0672b468 to 127.0.0.1:54098 2023-03-08 18:56:35,399 DEBUG [Listener at localhost/36281] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:35,400 DEBUG [Listener at localhost/36281] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-08 18:56:35,400 DEBUG [Listener at localhost/36281] util.JVMClusterUtil(257): Found active master hash=1257259383, stopped=false 2023-03-08 18:56:35,400 INFO [Listener at localhost/36281] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:56:35,402 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-08 18:56:35,402 INFO [Listener at localhost/36281] procedure2.ProcedureExecutor(629): Stopping 2023-03-08 18:56:35,402 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-08 18:56:35,402 DEBUG [Listener at localhost/36281] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4905b6f3 to 127.0.0.1:54098 2023-03-08 18:56:35,402 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:56:35,402 DEBUG [Listener at localhost/36281] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:35,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 18:56:35,403 INFO [Listener at localhost/36281] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase6.apache.org,46397,1678301621627' ***** 2023-03-08 18:56:35,403 INFO [Listener at localhost/36281] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-08 18:56:35,403 INFO [RS:0;jenkins-hbase6:46397] regionserver.HeapMemoryManager(220): Stopping 2023-03-08 18:56:35,403 INFO [RS:0;jenkins-hbase6:46397] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-08 18:56:35,403 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-08 18:56:35,403 INFO [RS:0;jenkins-hbase6:46397] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-08 18:56:35,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 18:56:35,404 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for d2ad469b39f67aef7d6b850cf65d8a4e 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 924cd1dc6b0366a92e4bc7c5a4762a13 2023-03-08 18:56:35,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing fef10155ee8ca5fb6840d90511945fb1, disabling compactions & flushes 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 8eb59701ce5a41a0784e567680dbb06b 2023-03-08 18:56:35,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 7c4e5be8c7bf7634fdec5582cd8f9f5e 2023-03-08 18:56:35,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for f3e546ab6e95b3c35d94bf0350799d5a 2023-03-08 18:56:35,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. after waiting 0 ms 2023-03-08 18:56:35,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 8336865ae81041d039911221d7ad790d 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 75fe5187a546ce484df6e7222327ef89 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 70855e40ae020380397c6a9b2ff4c892 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for b4625189889702b73dd8f5a775ba90f1 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for a3c6eb6683fb37966ee22d85c9e40cc1 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 8b7ffc3b5de37aaac329ae67ecaf3e9f 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 087e27baac917de6bad03c6444a74b6d 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for c33f16a1cf86167f9f09d4bdddce6467 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 40ac1522a08b03671fa59178494d47f1 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for bd9899deeb96650165ed7c0a6cf10aad 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for f9ff7d9a8ef2065b168b9dc4709311ac 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 8cccb8903f14f2a6b0b76a282ea1eb32 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 07707d9a61597d1f7ba6be12d211db21 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:56:35,405 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1145): stopping server jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:35,405 DEBUG [RS:0;jenkins-hbase6:46397] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x30bc36e3 to 127.0.0.1:54098 2023-03-08 18:56:35,405 DEBUG [RS:0;jenkins-hbase6:46397] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:35,406 INFO [RS:0;jenkins-hbase6:46397] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-08 18:56:35,406 INFO [RS:0;jenkins-hbase6:46397] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-08 18:56:35,406 INFO [RS:0;jenkins-hbase6:46397] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-08 18:56:35,406 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-08 18:56:35,407 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-08 18:56:35,407 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-08 18:56:35,407 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-08 18:56:35,407 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-08 18:56:35,407 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-08 18:56:35,407 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-08 18:56:35,407 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-08 18:56:35,408 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1505): Waiting on 07707d9a61597d1f7ba6be12d211db21, 087e27baac917de6bad03c6444a74b6d, 1588230740, 40ac1522a08b03671fa59178494d47f1, 70855e40ae020380397c6a9b2ff4c892, 75fe5187a546ce484df6e7222327ef89, 7c4e5be8c7bf7634fdec5582cd8f9f5e, 8336865ae81041d039911221d7ad790d, 8b7ffc3b5de37aaac329ae67ecaf3e9f, 8cccb8903f14f2a6b0b76a282ea1eb32, 8eb59701ce5a41a0784e567680dbb06b, 924cd1dc6b0366a92e4bc7c5a4762a13, a3c6eb6683fb37966ee22d85c9e40cc1, b4625189889702b73dd8f5a775ba90f1, bd9899deeb96650165ed7c0a6cf10aad, c33f16a1cf86167f9f09d4bdddce6467, d2ad469b39f67aef7d6b850cf65d8a4e, f3e546ab6e95b3c35d94bf0350799d5a, f9ff7d9a8ef2065b168b9dc4709311ac, fb8791aea8f6b5c60c5c17523aa003f3, fef10155ee8ca5fb6840d90511945fb1 2023-03-08 18:56:35,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithMonthlyPolicy/fef10155ee8ca5fb6840d90511945fb1/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-08 18:56:35,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:35,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for fef10155ee8ca5fb6840d90511945fb1: 2023-03-08 18:56:35,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1678301763975.fef10155ee8ca5fb6840d90511945fb1. 2023-03-08 18:56:35,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing d2ad469b39f67aef7d6b850cf65d8a4e, disabling compactions & flushes 2023-03-08 18:56:35,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:56:35,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:56:35,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. after waiting 0 ms 2023-03-08 18:56:35,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:56:35,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/d2ad469b39f67aef7d6b850cf65d8a4e/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 18:56:35,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:56:35,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for d2ad469b39f67aef7d6b850cf65d8a4e: 2023-03-08 18:56:35,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1678301745672.d2ad469b39f67aef7d6b850cf65d8a4e. 2023-03-08 18:56:35,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 924cd1dc6b0366a92e4bc7c5a4762a13, disabling compactions & flushes 2023-03-08 18:56:35,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:56:35,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:56:35,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. after waiting 0 ms 2023-03-08 18:56:35,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:56:35,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/924cd1dc6b0366a92e4bc7c5a4762a13/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 18:56:35,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:56:35,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 924cd1dc6b0366a92e4bc7c5a4762a13: 2023-03-08 18:56:35,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1678301717857.924cd1dc6b0366a92e4bc7c5a4762a13. 2023-03-08 18:56:35,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 8eb59701ce5a41a0784e567680dbb06b, disabling compactions & flushes 2023-03-08 18:56:35,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:56:35,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:56:35,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. after waiting 0 ms 2023-03-08 18:56:35,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:56:35,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/8eb59701ce5a41a0784e567680dbb06b/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 18:56:35,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:56:35,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 8eb59701ce5a41a0784e567680dbb06b: 2023-03-08 18:56:35,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1678301745672.8eb59701ce5a41a0784e567680dbb06b. 2023-03-08 18:56:35,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 7c4e5be8c7bf7634fdec5582cd8f9f5e, disabling compactions & flushes 2023-03-08 18:56:35,431 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:35,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:35,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. after waiting 0 ms 2023-03-08 18:56:35,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:35,434 INFO [regionserver/jenkins-hbase6:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-08 18:56:35,434 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/7c4e5be8c7bf7634fdec5582cd8f9f5e/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 18:56:35,435 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:35,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 7c4e5be8c7bf7634fdec5582cd8f9f5e: 2023-03-08 18:56:35,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1678301777769.7c4e5be8c7bf7634fdec5582cd8f9f5e. 2023-03-08 18:56:35,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing f3e546ab6e95b3c35d94bf0350799d5a, disabling compactions & flushes 2023-03-08 18:56:35,435 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:35,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:35,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. after waiting 0 ms 2023-03-08 18:56:35,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:35,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/f3e546ab6e95b3c35d94bf0350799d5a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 18:56:35,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:35,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for f3e546ab6e95b3c35d94bf0350799d5a: 2023-03-08 18:56:35,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1678301777769.f3e546ab6e95b3c35d94bf0350799d5a. 2023-03-08 18:56:35,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 8336865ae81041d039911221d7ad790d, disabling compactions & flushes 2023-03-08 18:56:35,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:56:35,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:56:35,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. after waiting 0 ms 2023-03-08 18:56:35,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:56:35,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithMonthlyPolicy/8336865ae81041d039911221d7ad790d/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-08 18:56:35,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:56:35,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 8336865ae81041d039911221d7ad790d: 2023-03-08 18:56:35,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1678301638525.8336865ae81041d039911221d7ad790d. 2023-03-08 18:56:35,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 75fe5187a546ce484df6e7222327ef89, disabling compactions & flushes 2023-03-08 18:56:35,447 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:56:35,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:56:35,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. after waiting 0 ms 2023-03-08 18:56:35,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:56:35,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/75fe5187a546ce484df6e7222327ef89/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-08 18:56:35,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:56:35,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 75fe5187a546ce484df6e7222327ef89: 2023-03-08 18:56:35,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678301681627.75fe5187a546ce484df6e7222327ef89. 2023-03-08 18:56:35,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 70855e40ae020380397c6a9b2ff4c892, disabling compactions & flushes 2023-03-08 18:56:35,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:56:35,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:56:35,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. after waiting 0 ms 2023-03-08 18:56:35,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:56:35,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/70855e40ae020380397c6a9b2ff4c892/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 18:56:35,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:56:35,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 70855e40ae020380397c6a9b2ff4c892: 2023-03-08 18:56:35,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1678301717857.70855e40ae020380397c6a9b2ff4c892. 2023-03-08 18:56:35,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing b4625189889702b73dd8f5a775ba90f1, disabling compactions & flushes 2023-03-08 18:56:35,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:56:35,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:56:35,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. after waiting 0 ms 2023-03-08 18:56:35,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:56:35,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/ns/testMinorCompaction/b4625189889702b73dd8f5a775ba90f1/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 18:56:35,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:56:35,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for b4625189889702b73dd8f5a775ba90f1: 2023-03-08 18:56:35,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1678301745672.b4625189889702b73dd8f5a775ba90f1. 2023-03-08 18:56:35,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing a3c6eb6683fb37966ee22d85c9e40cc1, disabling compactions & flushes 2023-03-08 18:56:35,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:56:35,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:56:35,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. after waiting 0 ms 2023-03-08 18:56:35,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:56:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/a3c6eb6683fb37966ee22d85c9e40cc1/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-08 18:56:35,462 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:56:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for a3c6eb6683fb37966ee22d85c9e40cc1: 2023-03-08 18:56:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1678301664893.a3c6eb6683fb37966ee22d85c9e40cc1. 2023-03-08 18:56:35,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 8b7ffc3b5de37aaac329ae67ecaf3e9f, disabling compactions & flushes 2023-03-08 18:56:35,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:35,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:35,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. after waiting 0 ms 2023-03-08 18:56:35,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:35,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionFromAdmin/8b7ffc3b5de37aaac329ae67ecaf3e9f/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 18:56:35,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 8b7ffc3b5de37aaac329ae67ecaf3e9f: 2023-03-08 18:56:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1678301777769.8b7ffc3b5de37aaac329ae67ecaf3e9f. 2023-03-08 18:56:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 087e27baac917de6bad03c6444a74b6d, disabling compactions & flushes 2023-03-08 18:56:35,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:56:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:56:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. after waiting 0 ms 2023-03-08 18:56:35,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:56:35,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testCompactionWithHFileLink/087e27baac917de6bad03c6444a74b6d/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 18:56:35,475 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:56:35,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 087e27baac917de6bad03c6444a74b6d: 2023-03-08 18:56:35,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1678301717857.087e27baac917de6bad03c6444a74b6d. 2023-03-08 18:56:35,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing c33f16a1cf86167f9f09d4bdddce6467, disabling compactions & flushes 2023-03-08 18:56:35,475 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:56:35,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:56:35,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. after waiting 0 ms 2023-03-08 18:56:35,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:56:35,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMinorCompactionWithWeeklyPolicy/c33f16a1cf86167f9f09d4bdddce6467/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 18:56:35,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:56:35,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for c33f16a1cf86167f9f09d4bdddce6467: 2023-03-08 18:56:35,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1678301652467.c33f16a1cf86167f9f09d4bdddce6467. 2023-03-08 18:56:35,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 40ac1522a08b03671fa59178494d47f1, disabling compactions & flushes 2023-03-08 18:56:35,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:56:35,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:56:35,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. after waiting 0 ms 2023-03-08 18:56:35,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:56:35,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/40ac1522a08b03671fa59178494d47f1/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-08 18:56:35,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:56:35,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 40ac1522a08b03671fa59178494d47f1: 2023-03-08 18:56:35,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1678301664893.40ac1522a08b03671fa59178494d47f1. 2023-03-08 18:56:35,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing bd9899deeb96650165ed7c0a6cf10aad, disabling compactions & flushes 2023-03-08 18:56:35,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:56:35,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:56:35,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. after waiting 0 ms 2023-03-08 18:56:35,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:56:35,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing bd9899deeb96650165ed7c0a6cf10aad 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-08 18:56:35,490 INFO [regionserver/jenkins-hbase6:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-08 18:56:35,608 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1505): Waiting on 07707d9a61597d1f7ba6be12d211db21, 1588230740, 8cccb8903f14f2a6b0b76a282ea1eb32, bd9899deeb96650165ed7c0a6cf10aad, f9ff7d9a8ef2065b168b9dc4709311ac, fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:56:35,808 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1505): Waiting on 07707d9a61597d1f7ba6be12d211db21, 1588230740, 8cccb8903f14f2a6b0b76a282ea1eb32, bd9899deeb96650165ed7c0a6cf10aad, f9ff7d9a8ef2065b168b9dc4709311ac, fb8791aea8f6b5c60c5c17523aa003f3 2023-03-08 18:56:35,821 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/.tmp/info/d2355ca5a5664ff1b992636d71360068 2023-03-08 18:56:35,825 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d2355ca5a5664ff1b992636d71360068 2023-03-08 18:56:35,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad/.tmp/info/36c2a8c157fa4030bc6c09d04939715b 2023-03-08 18:56:35,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad/.tmp/info/36c2a8c157fa4030bc6c09d04939715b as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad/info/36c2a8c157fa4030bc6c09d04939715b 2023-03-08 18:56:35,904 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad/info/36c2a8c157fa4030bc6c09d04939715b, entries=3, sequenceid=7, filesize=4.8 K 2023-03-08 18:56:35,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for bd9899deeb96650165ed7c0a6cf10aad in 424ms, sequenceid=7, compaction requested=false 2023-03-08 18:56:35,908 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/namespace/bd9899deeb96650165ed7c0a6cf10aad/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-08 18:56:35,908 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:56:35,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for bd9899deeb96650165ed7c0a6cf10aad: 2023-03-08 18:56:35,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1678301624714.bd9899deeb96650165ed7c0a6cf10aad. 2023-03-08 18:56:35,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing f9ff7d9a8ef2065b168b9dc4709311ac, disabling compactions & flushes 2023-03-08 18:56:35,909 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:56:35,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:56:35,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. after waiting 0 ms 2023-03-08 18:56:35,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:56:35,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyPolicy/f9ff7d9a8ef2065b168b9dc4709311ac/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 18:56:35,911 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:56:35,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for f9ff7d9a8ef2065b168b9dc4709311ac: 2023-03-08 18:56:35,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1678301669494.f9ff7d9a8ef2065b168b9dc4709311ac. 2023-03-08 18:56:35,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 8cccb8903f14f2a6b0b76a282ea1eb32, disabling compactions & flushes 2023-03-08 18:56:35,912 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:56:35,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:56:35,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. after waiting 0 ms 2023-03-08 18:56:35,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:56:35,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8cccb8903f14f2a6b0b76a282ea1eb32/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 18:56:35,914 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:56:35,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 8cccb8903f14f2a6b0b76a282ea1eb32: 2023-03-08 18:56:35,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678301625985.8cccb8903f14f2a6b0b76a282ea1eb32. 2023-03-08 18:56:35,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 07707d9a61597d1f7ba6be12d211db21, disabling compactions & flushes 2023-03-08 18:56:35,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:56:35,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:56:35,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. after waiting 0 ms 2023-03-08 18:56:35,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:56:35,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testScannerOnBulkLoadRefHFiles/07707d9a61597d1f7ba6be12d211db21/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-08 18:56:35,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:56:35,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 07707d9a61597d1f7ba6be12d211db21: 2023-03-08 18:56:35,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1678301664893.07707d9a61597d1f7ba6be12d211db21. 2023-03-08 18:56:35,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing fb8791aea8f6b5c60c5c17523aa003f3, disabling compactions & flushes 2023-03-08 18:56:35,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:56:35,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:56:35,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. after waiting 0 ms 2023-03-08 18:56:35,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:56:35,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fb8791aea8f6b5c60c5c17523aa003f3/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-08 18:56:35,919 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:56:35,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for fb8791aea8f6b5c60c5c17523aa003f3: 2023-03-08 18:56:35,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678301697529.fb8791aea8f6b5c60c5c17523aa003f3. 2023-03-08 18:56:36,009 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-08 18:56:36,065 DEBUG [regionserver/jenkins-hbase6:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 18:56:36,209 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-08 18:56:36,232 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/.tmp/rep_barrier/477b123563e3419e934f9282d334348d 2023-03-08 18:56:36,236 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 477b123563e3419e934f9282d334348d 2023-03-08 18:56:36,382 INFO [regionserver/jenkins-hbase6:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-08 18:56:36,382 INFO [regionserver/jenkins-hbase6:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-08 18:56:36,409 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-03-08 18:56:36,409 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-03-08 18:56:36,409 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-08 18:56:36,610 DEBUG [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-08 18:56:36,642 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/.tmp/table/2991a5e636c54a478a85998a654e6f66 2023-03-08 18:56:36,645 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2991a5e636c54a478a85998a654e6f66 2023-03-08 18:56:36,645 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/.tmp/info/d2355ca5a5664ff1b992636d71360068 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/info/d2355ca5a5664ff1b992636d71360068 2023-03-08 18:56:36,649 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d2355ca5a5664ff1b992636d71360068 2023-03-08 18:56:36,649 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/info/d2355ca5a5664ff1b992636d71360068, entries=226, sequenceid=130, filesize=37.9 K 2023-03-08 18:56:36,649 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/.tmp/rep_barrier/477b123563e3419e934f9282d334348d as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/rep_barrier/477b123563e3419e934f9282d334348d 2023-03-08 18:56:36,653 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 477b123563e3419e934f9282d334348d 2023-03-08 18:56:36,653 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/rep_barrier/477b123563e3419e934f9282d334348d, entries=1, sequenceid=130, filesize=5.0 K 2023-03-08 18:56:36,653 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/.tmp/table/2991a5e636c54a478a85998a654e6f66 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/table/2991a5e636c54a478a85998a654e6f66 2023-03-08 18:56:36,656 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2991a5e636c54a478a85998a654e6f66 2023-03-08 18:56:36,656 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/table/2991a5e636c54a478a85998a654e6f66, entries=27, sequenceid=130, filesize=6.6 K 2023-03-08 18:56:36,658 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1251ms, sequenceid=130, compaction requested=false 2023-03-08 18:56:36,661 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-08 18:56:36,662 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-08 18:56:36,662 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-08 18:56:36,662 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-08 18:56:36,662 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-08 18:56:36,810 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1171): stopping server jenkins-hbase6.apache.org,46397,1678301621627; all regions closed. 2023-03-08 18:56:36,816 DEBUG [RS:0;jenkins-hbase6:46397] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/oldWALs 2023-03-08 18:56:36,816 INFO [RS:0;jenkins-hbase6:46397] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase6.apache.org%2C46397%2C1678301621627.meta:.meta(num 1678301624455) 2023-03-08 18:56:36,820 DEBUG [RS:0;jenkins-hbase6:46397] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/oldWALs 2023-03-08 18:56:36,820 INFO [RS:0;jenkins-hbase6:46397] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase6.apache.org%2C46397%2C1678301621627:(num 1678301623546) 2023-03-08 18:56:36,820 DEBUG [RS:0;jenkins-hbase6:46397] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:36,820 INFO [RS:0;jenkins-hbase6:46397] regionserver.LeaseManager(133): Closed leases 2023-03-08 18:56:36,820 INFO [RS:0;jenkins-hbase6:46397] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase6:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-08 18:56:36,820 INFO [regionserver/jenkins-hbase6:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-08 18:56:36,821 INFO [RS:0;jenkins-hbase6:46397] ipc.NettyRpcServer(158): Stopping server on /172.31.2.20:46397 2023-03-08 18:56:36,829 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 18:56:36,829 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase6.apache.org,46397,1678301621627 2023-03-08 18:56:36,829 ERROR [Listener at localhost/36281-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@141cf079 rejected from java.util.concurrent.ThreadPoolExecutor@10237622[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 18:56:36,829 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 18:56:36,829 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase6.apache.org,46397,1678301621627] 2023-03-08 18:56:36,829 ERROR [Listener at localhost/36281-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@17524c39 rejected from java.util.concurrent.ThreadPoolExecutor@10237622[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 18:56:36,830 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase6.apache.org,46397,1678301621627; numProcessing=1 2023-03-08 18:56:36,832 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase6.apache.org,46397,1678301621627 already deleted, retry=false 2023-03-08 18:56:36,832 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase6.apache.org,46397,1678301621627 expired; onlineServers=0 2023-03-08 18:56:36,832 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase6.apache.org,44349,1678301619149' ***** 2023-03-08 18:56:36,832 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-08 18:56:36,832 DEBUG [M:0;jenkins-hbase6:44349] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@bcd4b4c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase6.apache.org/172.31.2.20:0 2023-03-08 18:56:36,832 INFO [M:0;jenkins-hbase6:44349] regionserver.HRegionServer(1145): stopping server jenkins-hbase6.apache.org,44349,1678301619149 2023-03-08 18:56:36,832 INFO [M:0;jenkins-hbase6:44349] regionserver.HRegionServer(1171): stopping server jenkins-hbase6.apache.org,44349,1678301619149; all regions closed. 2023-03-08 18:56:36,832 DEBUG [M:0;jenkins-hbase6:44349] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 18:56:36,833 DEBUG [M:0;jenkins-hbase6:44349] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-08 18:56:36,833 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-08 18:56:36,833 DEBUG [M:0;jenkins-hbase6:44349] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-08 18:56:36,833 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.small.0-1678301623177] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.small.0-1678301623177,5,FailOnTimeoutGroup] 2023-03-08 18:56:36,833 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.large.0-1678301623176] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.large.0-1678301623176,5,FailOnTimeoutGroup] 2023-03-08 18:56:36,833 INFO [M:0;jenkins-hbase6:44349] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-08 18:56:36,833 INFO [M:0;jenkins-hbase6:44349] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-08 18:56:36,833 INFO [M:0;jenkins-hbase6:44349] hbase.ChoreService(369): Chore service for: master/jenkins-hbase6:0 had [] on shutdown 2023-03-08 18:56:36,834 DEBUG [M:0;jenkins-hbase6:44349] master.HMaster(1512): Stopping service threads 2023-03-08 18:56:36,834 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-08 18:56:36,834 INFO [M:0;jenkins-hbase6:44349] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-08 18:56:36,834 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 18:56:36,834 ERROR [M:0;jenkins-hbase6:44349] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-08 18:56:36,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 18:56:36,834 INFO [M:0;jenkins-hbase6:44349] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-08 18:56:36,835 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-08 18:56:36,835 DEBUG [M:0;jenkins-hbase6:44349] zookeeper.ZKUtil(398): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-08 18:56:36,835 WARN [M:0;jenkins-hbase6:44349] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-08 18:56:36,835 INFO [M:0;jenkins-hbase6:44349] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-08 18:56:36,835 INFO [M:0;jenkins-hbase6:44349] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-08 18:56:36,835 DEBUG [M:0;jenkins-hbase6:44349] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-08 18:56:36,835 INFO [M:0;jenkins-hbase6:44349] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 18:56:36,835 DEBUG [M:0;jenkins-hbase6:44349] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 18:56:36,835 DEBUG [M:0;jenkins-hbase6:44349] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-08 18:56:36,836 DEBUG [M:0;jenkins-hbase6:44349] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 18:56:36,836 INFO [M:0;jenkins-hbase6:44349] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.81 KB heapSize=598.46 KB 2023-03-08 18:56:36,930 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 18:56:36,930 INFO [RS:0;jenkins-hbase6:46397] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase6.apache.org,46397,1678301621627; zookeeper connection closed. 2023-03-08 18:56:36,931 ERROR [Listener at localhost/36281-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6fa70151 rejected from java.util.concurrent.ThreadPoolExecutor@10237622[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 18:56:36,931 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@66594f6b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@66594f6b 2023-03-08 18:56:36,931 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): regionserver:46397-0x101d3e794b30001, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 18:56:36,931 ERROR [Listener at localhost/36281-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@30b8a411 rejected from java.util.concurrent.ThreadPoolExecutor@10237622[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 18:56:36,931 INFO [Listener at localhost/36281] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-08 18:56:37,242 INFO [M:0;jenkins-hbase6:44349] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.81 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/183572e0e4764a978ee80b84650468e3 2023-03-08 18:56:37,246 INFO [M:0;jenkins-hbase6:44349] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 183572e0e4764a978ee80b84650468e3 2023-03-08 18:56:37,246 DEBUG [M:0;jenkins-hbase6:44349] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/183572e0e4764a978ee80b84650468e3 as hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/183572e0e4764a978ee80b84650468e3 2023-03-08 18:56:37,249 INFO [M:0;jenkins-hbase6:44349] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 183572e0e4764a978ee80b84650468e3 2023-03-08 18:56:37,249 INFO [M:0;jenkins-hbase6:44349] regionserver.HStore(1080): Added hdfs://localhost:35703/user/jenkins/test-data/96f42769-05b9-d799-ea9d-089f76a74709/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/183572e0e4764a978ee80b84650468e3, entries=123, sequenceid=1005, filesize=33.3 K 2023-03-08 18:56:37,250 INFO [M:0;jenkins-hbase6:44349] regionserver.HRegion(2948): Finished flush of dataSize ~512.81 KB/525116, heapSize ~598.45 KB/612808, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 414ms, sequenceid=1005, compaction requested=false 2023-03-08 18:56:37,250 INFO [M:0;jenkins-hbase6:44349] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 18:56:37,250 DEBUG [M:0;jenkins-hbase6:44349] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 18:56:37,252 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-08 18:56:37,252 INFO [M:0;jenkins-hbase6:44349] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-08 18:56:37,253 INFO [M:0;jenkins-hbase6:44349] ipc.NettyRpcServer(158): Stopping server on /172.31.2.20:44349 2023-03-08 18:56:37,255 DEBUG [M:0;jenkins-hbase6:44349] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase6.apache.org,44349,1678301619149 already deleted, retry=false 2023-03-08 18:56:37,359 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 18:56:37,359 INFO [M:0;jenkins-hbase6:44349] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase6.apache.org,44349,1678301619149; zookeeper connection closed. 2023-03-08 18:56:37,359 ERROR [Listener at localhost/36281-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@25f25783 rejected from java.util.concurrent.ThreadPoolExecutor@4de62f4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 18:56:37,359 DEBUG [Listener at localhost/36281-EventThread] zookeeper.ZKWatcher(600): master:44349-0x101d3e794b30000, quorum=127.0.0.1:54098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 18:56:37,359 ERROR [Listener at localhost/36281-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5f9d37eb rejected from java.util.concurrent.ThreadPoolExecutor@4de62f4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 18:56:37,361 WARN [Listener at localhost/36281] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-08 18:56:37,367 INFO [Listener at localhost/36281] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-08 18:56:37,474 WARN [BP-475303163-172.31.2.20-1678301615763 heartbeating to localhost/127.0.0.1:35703] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-08 18:56:37,474 WARN [BP-475303163-172.31.2.20-1678301615763 heartbeating to localhost/127.0.0.1:35703] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-475303163-172.31.2.20-1678301615763 (Datanode Uuid d2e0e9f3-5706-4352-ae84-a5bb1a722721) service to localhost/127.0.0.1:35703 2023-03-08 18:56:37,476 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/cluster_8eed2209-15fd-42d6-1e88-8ecffcfc5d58/dfs/data/data1/current/BP-475303163-172.31.2.20-1678301615763] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-08 18:56:37,476 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5508e52d-c514-f196-3bec-af2d5bfd7dac/cluster_8eed2209-15fd-42d6-1e88-8ecffcfc5d58/dfs/data/data2/current/BP-475303163-172.31.2.20-1678301615763] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-08 18:56:37,504 INFO [Listener at localhost/36281] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-08 18:56:37,634 INFO [Listener at localhost/36281] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-08 18:56:37,718 INFO [Listener at localhost/36281] hbase.HBaseTestingUtility(1293): Minicluster is down